-- Logs begin at Fri 2022-06-24 08:57:35 UTC, end at Fri 2022-06-24 10:00:57 UTC. -- Jun 24 09:27:39.290396 dv-cinder-803823-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 09:27:39.302194 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 09:27:39 2022] *** Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: nodename: dv-cinder-803823-13-iscsi Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: machine: x86_64 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: clock source: unix Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: pcre jit disabled Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: detected number of CPU cores: 4 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: current working directory: / Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: detected binary path: /usr/bin/uwsgi-core Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: your processes number limit is 47680 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: your memory page size is 4096 bytes Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: detected max file descriptor number: 2048 Jun 24 09:27:39.304302 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: lock engine: pthread robust mutexes Jun 24 09:27:39.305080 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: thunder lock: enabled Jun 24 09:27:39.305080 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 09:27:39.305080 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 09:27:39.347963 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: Python main interpreter initialized at 0x557cadeb09c0 Jun 24 09:27:39.347963 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: python threads support enabled Jun 24 09:27:39.347963 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: your server socket listen backlog is limited to 100 connections Jun 24 09:27:39.347963 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: your mercy for graceful operations on workers is 90 seconds Jun 24 09:27:39.348316 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 09:27:39.348377 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: *** Operational MODE: preforking *** Jun 24 09:27:39.348644 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: *** uWSGI is running in multiple interpreter mode *** Jun 24 09:27:39.348644 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: spawned uWSGI master process (pid: 80351) Jun 24 09:27:39.348859 dv-cinder-803823-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 09:27:39.349363 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: spawned uWSGI worker 1 (pid: 80358, cores: 1) Jun 24 09:27:39.349802 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: spawned uWSGI worker 2 (pid: 80359, cores: 1) Jun 24 09:27:39.349968 dv-cinder-803823-13-iscsi devstack@keystone.service[80351]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 09:27:44.726017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:27:44.726017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: from cryptography.utils import int_from_bytes Jun 24 09:27:44.726870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:27:44.726870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 24 09:27:44.727342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:27:44.727342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: from cryptography.utils import int_from_bytes Jun 24 09:27:44.727448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 09:27:44.727448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 24 09:27:44.776493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 09:27:44.776688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 09:27:44.776836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] command line args: [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 09:27:44.777132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 09:27:44.777332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ================================================================================ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 09:27:44.777565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] admin_token = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.777809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_dir = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.778005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.778177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_source = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.778396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] debug = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.778514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.778905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.779052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.779358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.779603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.779840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.779968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_config_append = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.780181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.780349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_dir = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.780529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.780747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.780991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.781234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.781477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.781650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.781826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.782064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.782311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.782542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.782777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.782989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.783245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.783490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.783733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.784003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.784500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.784682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] publish_errors = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.784911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.785143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.785321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.785563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.785793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.786026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] standard_threads = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.786256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.786494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.786671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.786846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_journal = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.787212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_json = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.787304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_stderr = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.787415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_syslog = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.787645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.787903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.788137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.788392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.788631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.788877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.789116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.789358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.789592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.789835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.790080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.790331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.790563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.790808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.791042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.791287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 09:27:44.791381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.791487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 09:27:44.791570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.791684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] command line args: [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 09:27:44.791758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.791974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 09:27:44.792050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.792157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ================================================================================ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 09:27:44.792240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.792415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.792489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.792663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_dir = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.792742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.792854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.792929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.793038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_source = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.793162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.793247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] debug = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.793391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.793474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.793636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.793714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.793835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.793906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.794052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.794127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.794290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.794362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.794520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.794595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.794703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_config_append = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.794783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.794883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.794992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.795114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_dir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.795246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.795329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.795474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.795549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.795716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.795799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.795956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.796030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.796191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.796266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.796366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.796441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.796544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.796647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.796788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.796868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.797034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.797110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.797270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.797350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.797503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.797578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.797737 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.797812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.797963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.798037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.798197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.798278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.798440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.798514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.798665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.798746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.798902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.799161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.799373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.799552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.799629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] publish_errors = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.799688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.799842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.799916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.799977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.800034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.800090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.800269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.800364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.800498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.800569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.800734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.800816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] standard_threads = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.800964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.801046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.801196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.801269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.801375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.801447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.801555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_journal = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.801642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.801719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_json = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.801876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.801950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_stderr = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.802058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_syslog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.802142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.802296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 09:27:44.802375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.802573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.802687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.802848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.802970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.803860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.804894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.805841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.806945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.807923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.808947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.809929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.810957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.811986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.external = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.812898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.password = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.token = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.813890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.814945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.815918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.816933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.817922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.818913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.819892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.820991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.821932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.822977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.823927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.824864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.825958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.826940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.external = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.827975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.password = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.token = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.828907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.829851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.830928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.831971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.832945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833707 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.833987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.834966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.835948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.836962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.837918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.838854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.839973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.840997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.841948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.842898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.843868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.844972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.845977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.846926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.847913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.848856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.849791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.850977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.851954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.852944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.853907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.854999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.855107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.855264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.855340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.855510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.855585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.856980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.857939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.858912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.859899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.860599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.861985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.862935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.863097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.863177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.863277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 09:27:44.863365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.863582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.863832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.864082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.864331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.864743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.864981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.865268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.865520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.865770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.866019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.866258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.866516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.866773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.867012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.867276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.867511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.867745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.867997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.868252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.868508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.868749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.868987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.869221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.869459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.869699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.869946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.870183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.870440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.870687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.870925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.871198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.871446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.871693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.871932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.872171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.872406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.872653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.872901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.873132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.873377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.873674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.873934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.874180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.874427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.874666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.874907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.875166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.875408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.875645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.875884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.876494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.876729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.876974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.877210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.877458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.877694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.877957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.878189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.878439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.878675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 09:27:44.878848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 09:27:44.894895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.895148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.895386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.904951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.905051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.905246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.905477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.914139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.932667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.932894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.933935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:44.934168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.078192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.078673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.127715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.127934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.132405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.132655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.191449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.191601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.191671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.191864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.191990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.192685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.217277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.217473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.217766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.217929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.218828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.219015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.219283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.229844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.230366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.242690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.242900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.243964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.244051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.244223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.244406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.244650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 09:27:45.261791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.261993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:27:45.262345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.262467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.262556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:27:45.262635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:27:45.262782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.262998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.263146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.263215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 09:27:45.263309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.263457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.263547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.263751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.264007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.264261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.274443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.274718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.274965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.275215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.275338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.275459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.275757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.276040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.283607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.284402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.286886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.287535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.287535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.287733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.287825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.288895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.289895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.290165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.290429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.316063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.317201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.317668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.317943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.319011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.319315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.323352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.323736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.323994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.324617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.325002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.325269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.331387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.331621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.332655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.332941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.336385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.336661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.336935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.337211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.337482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.337606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.337750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.337876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.337985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.338077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.338251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.338347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.338589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.338860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.339115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.339397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.357007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.357292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.358280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.358558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.361980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.363252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.365676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.366071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.366951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.367376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.371099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.372444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.374772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.376514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.378816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.379104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.379420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.379699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.379978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.380209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.380324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.380535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.380854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.381145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.381527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.381809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.399506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.399773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.400037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.400303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.401782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.402078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.402359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.402630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.408248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.409947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.410225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.410553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.412319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.412571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.413527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.413827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.414124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.414430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.414773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.415864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.416163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.416462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.416764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.417043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.428362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.428651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.428932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.429213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.430541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.430783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.430907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.431041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.431143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.431374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.431498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.432955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.433251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.433540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.449301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.451598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.452619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.454886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.455954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.457664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.458217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.459396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.459696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.459980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.460181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.461844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.462116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.462401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.467287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.468895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.469768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.471418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.472185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.472472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.472864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.473137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.474668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.474970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.475385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.475674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.482280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.482565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.482840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.484810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.485101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.485380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.490986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.491287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.491565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.491850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.492130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.492507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.492787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 09:27:45.493570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.493945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.494723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.495005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.495315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.495590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.495861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.496137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.496405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.496689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.496966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 09:27:45.533246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557cadeb09c0 pid: 80359 (default app) Jun 24 09:27:45.536688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec639-bd70-44ca-b8f1-cacec8a60f8e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:45.536814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557cadeb09c0 pid: 80358 (default app) Jun 24 09:27:45.537095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec639-bd70-44ca-b8f1-cacec8a60f8e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:45.537386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec639-bd70-44ca-b8f1-cacec8a60f8e None None] PATH_INFO: `/v3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:45.541623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Fri Jun 24 09:27:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:27:53.326552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:53.327218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:53.327218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:53.331512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:27:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:27:53.338732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:53.339562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:53.339679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:53.412292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: INFO dbcounter [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] Registered counter for database keystone Jun 24 09:27:53.436941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG oslo_db.sqlalchemy.engines [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80359) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 09:27:53.447586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writer thread running {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 09:27:53.634786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG passlib.handlers.bcrypt [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80359) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 09:27:53.635114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG passlib.handlers.bcrypt [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80359) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 09:27:53.690951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:27:53.692567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:53.829935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:27:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 09:27:53.990474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:54.067990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: INFO dbcounter [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] Registered counter for database keystone Jun 24 09:27:54.090170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG oslo_db.sqlalchemy.engines [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80358) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 09:27:54.098634 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writer thread running {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 09:27:54.219304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf30868-68cb-4c9b-a1bd-77034aaa7ef4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:54.224901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:54.225567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:54.225858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:54.226133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:54.235420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:54.287853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING keystone.common.rbac_enforcer.enforcer [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 09:27:54.288128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d33b1-017b-4c9d-9268-d67417b30a7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:54.298583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Fri Jun 24 09:27:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:27:54.305338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:54.321406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973b6bb-b79e-4f0e-a1f3-30803bce5055 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:54.327707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:54.328871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:54.329159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:54.329439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] PATH_INFO: `/v3/projects/46a0c5caeb0244aaa5e89844285d900b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:54.333850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] RBAC: Authorizing `identity:get_project(project_id=46a0c5caeb0244aaa5e89844285d900b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:54.385282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING keystone.common.rbac_enforcer.enforcer [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 09:27:54.385851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a33cf-c429-462f-b5c4-e064946a80a4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:54.390285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Fri Jun 24 09:27:54 2022] GET /identity/v3/projects/46a0c5caeb0244aaa5e89844285d900b => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:27:56.435778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524f4e-885b-4849-a6c6-8092d873d4ef None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:56.436346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524f4e-885b-4849-a6c6-8092d873d4ef None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:56.436346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524f4e-885b-4849-a6c6-8092d873d4ef None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:56.443113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:27:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:27:56.452285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:56.452699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:56.452879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:56.479058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:27:56.481579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:56.494766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:27:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 09:27:56.518369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6524f4e-885b-4849-a6c6-8092d873d4ef None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:56.535235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6524f4e-885b-4849-a6c6-8092d873d4ef None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:56.541976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:56.542621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:56.542909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:56.543210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:56.552103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:56.554006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6c219-240f-4f98-ad90-4b18323b2e4a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:56.563150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Fri Jun 24 09:27:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:27:56.569474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:56.584378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0dd1-d7ad-452e-8b38-639aa9b74113 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:56.590269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:56.591179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:56.591473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:56.591733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:56.593263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:56.595233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ac92d-a30e-4efc-8d75-b2c149900324 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:56.596840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Fri Jun 24 09:27:56 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:27:58.768762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331630-71b0-4dea-8f65-6551ab62ce30 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:58.769279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331630-71b0-4dea-8f65-6551ab62ce30 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:58.769279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331630-71b0-4dea-8f65-6551ab62ce30 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:58.769711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7b05-d366-48fe-a537-15a919a48406 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:58.770131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7b05-d366-48fe-a537-15a919a48406 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:58.770265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf7b05-d366-48fe-a537-15a919a48406 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:58.770693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:27:58.771913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:27:58.776706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:58.776860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:58.776994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:58.777233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:58.777329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:58.777515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:58.801037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:27:58.802657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:58.827904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80358) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 09:27:58.828323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80358) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 09:27:58.873715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:27:58.875435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:27:58.889985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 7/12] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:27:58.914081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:58.940146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:58.982020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 6/13] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:27:59.005152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.023883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d75a4-92b8-4f0c-a12f-5d5a2e930b27 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.060533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.061608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.061891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.062165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.073318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:27:59.084770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:27:59.085676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:27:59.086482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 8/14] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:27:59.092030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.112681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2b9a4-6f60-4786-9379-7e0c160fd3d5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.115265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.155610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.156536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.156827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.157103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.182730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.210169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23a42c-7cfe-427a-aa47-becfdb4b9472 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.229750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.231316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.245467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.246155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.246434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.246725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.247513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:27:59.247818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.249205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.262106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 9/15] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:27:59.267875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.330252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 7/16] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:27:59.335945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.338648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.368178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae99278-7f8b-447a-8332-e2d94c774e1e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.402117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.402997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.403329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.403593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.404825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.406303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.407984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 10/17] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:27:59.414049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.428586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.437630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b92bd-5a57-4ec7-9bdc-7fc79cb6bb22 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.471999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.472808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.473146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.473431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.492161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.493659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:27:59.515127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:27:59.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:27:59.516747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:27:59.516747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:27:59.516747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:27:59.516747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:27:59.516747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 8/18] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:27:59.522131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.529560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101037d8-84b9-42d4-bf25-e248d39fd451 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.571178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.571988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.572250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.572514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.573723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.575227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.576952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 11/19] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:27:59.579505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.583321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:27:59.623881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471b242-20dc-47e6-be87-9499f1d627a6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.659291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.659963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.660242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.660516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.661258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:27:59.661365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:27:59.661563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.663326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e5298-64d1-45de-b96a-2ccb607ff053 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.680828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3af8c0-0a58-4cc2-958f-c2b15ba7cd79 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:27:59.680994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 9/20] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:27:59.719217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:27:59.719893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:27:59.720175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:27:59.720449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:27:59.721298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.domain.id=default, user.id=0fdc8f07afea4aa1a81df36db46bb225) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:27:59.721603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:27:59.723116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4111d-7bdd-4604-82a1-c20aff21cf44 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:27:59.731124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 12/21] 10.222.0.65 () {56 vars in 1322 bytes} [Fri Jun 24 09:27:59 2022] GET /identity/v3/role_assignments?user.id=0fdc8f07afea4aa1a81df36db46bb225&scope.domain.id=default&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:02.025189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48f00-bc1a-484f-a562-32608be6af98 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.026022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48f00-bc1a-484f-a562-32608be6af98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.026471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf48f00-bc1a-484f-a562-32608be6af98 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.028095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 10/22] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:02.037011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.037982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.038448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.060269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:02.061792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:02.076784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106f67-653d-4e4a-a766-c4ecc5a3e3b3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.077396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106f67-653d-4e4a-a766-c4ecc5a3e3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.077859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106f67-653d-4e4a-a766-c4ecc5a3e3b3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.079475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 11/23] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:02.085697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.086206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.086669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.110904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:02.113465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:02.186375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 13/24] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:02.218609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:02.267038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 12/25] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:02.318021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:02.329493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:02.356748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605f4c8-8f01-427c-989d-c7dfae76a318 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:02.393322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:02.394411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.394836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.395276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.406600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:02.408428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:28:02.427710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:02.428581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:02.429360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:28:02.429360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:28:02.429360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:28:02.429360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:02.431041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:02.431599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 14/26] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:02.437396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:02.581111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb208f59-209c-4958-aac3-0d2a6962c633 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:02.589764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefd4ec-9854-4aef-ac8e-475bfbf4b7de None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:02.635542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:02.646181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:02.655369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.655908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.656383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.657389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:02.657901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:02.659620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875beb-a521-490d-ab4c-ecfc4088c8f7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:02.671765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:02.672863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.673361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.673832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.678867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 15/27] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:02.680261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e326d1f-27ff-4462-8fe4-eeb1360770b2 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.680553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e326d1f-27ff-4462-8fe4-eeb1360770b2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.680832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e326d1f-27ff-4462-8fe4-eeb1360770b2 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.682072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 16/28] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:02.687962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e326d1f-27ff-4462-8fe4-eeb1360770b2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:02.699646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7e342673-f4b5-435c-9c05-19ebec85bbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:02.713849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:02.714697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:02.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:02.720827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 13/29] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:02.724592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.726077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.727316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.772499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:02.774833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:02.879232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 14/30] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:02.880032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:02.881790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e326d1f-27ff-4462-8fe4-eeb1360770b2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:02.917629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:02.918689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:02.919197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:02.919683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:02.920586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:02.921926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:02.923204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:02.994718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bea3e-7286-4232-b9c8-1fe482773a57 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.030795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.031635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.031857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.032144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.032960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:03.033275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.034761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390735-52d7-47ef-a084-ec0f10511741 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.047757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 15/31] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:03.049075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4bc4-c3c0-47a2-8980-7c6033a0690b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.049367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4bc4-c3c0-47a2-8980-7c6033a0690b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.054241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f4bc4-c3c0-47a2-8980-7c6033a0690b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.059569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 16/32] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:03.059569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f4bc4-c3c0-47a2-8980-7c6033a0690b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.077828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 17/33] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:28:02 2022] POST /identity/v3/projects => generated 301 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:03.079703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.127517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:03.166914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:03.172349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f4bc4-c3c0-47a2-8980-7c6033a0690b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.206846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d02e-a47b-4d45-bdc1-d531d5f535ef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.224937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.225645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.225938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.226221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.227185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.228941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.250483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.252253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.252725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.253187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.254744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.256691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4326db-ff46-4bac-a1ab-6d1663235945 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.258560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 18/34] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:03 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:03.260159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.260668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.261131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.279812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:03.281651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:03.365374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 19/35] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:03.365374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.392579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 17/36] 10.222.0.65 () {60 vars in 1131 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/roles => generated 225 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:03.402219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:03.404281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.482301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e46f48-2f54-4885-887c-f5e76b751c49 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.525732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30772-1ac9-468b-ab82-ed29878859dd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.568078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.569206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.569691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.570187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.571229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.573045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.608732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 18/37] 10.222.0.65 () {60 vars in 1131 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/roles => generated 219 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:03.643523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.644720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.645085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.645345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.647301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.649136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51101702-243c-49c5-9ba9-b257f82b18da None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.651147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 20/38] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:03.657000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.659434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5f6c-ba41-4d8a-86cf-fcffe97ea7ef None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.659958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5f6c-ba41-4d8a-86cf-fcffe97ea7ef None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.660361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5f6c-ba41-4d8a-86cf-fcffe97ea7ef None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.662597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 21/39] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:03.667933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532509b-d3bb-4872-bc06-0d02cabc482b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.668318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532509b-d3bb-4872-bc06-0d02cabc482b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.668698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532509b-d3bb-4872-bc06-0d02cabc482b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.688799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5532509b-d3bb-4872-bc06-0d02cabc482b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:03.690285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5532509b-d3bb-4872-bc06-0d02cabc482b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:03.693732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:03.788477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 22/40] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:03.790436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655267b-8682-4e3d-9af1-278bcd30dab7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.790803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655267b-8682-4e3d-9af1-278bcd30dab7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.795574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655267b-8682-4e3d-9af1-278bcd30dab7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.797259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 23/41] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:03.805550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.806057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.806823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.821264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b554b44-a2ed-4f3c-bedd-2cf07a9db606 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:03.828191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:03.829420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:03.862142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:03.864231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:03.865262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:03.865564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:03.865845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] PATH_INFO: `/v3/domains/default/users/0fdc8f07afea4aa1a81df36db46bb225/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:03.869325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0fdc8f07afea4aa1a81df36db46bb225, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:03.870820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:03.887777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 19/42] 10.222.0.65 () {58 vars in 1361 bytes} [Fri Jun 24 09:28:03 2022] PUT /identity/v3/domains/default/users/0fdc8f07afea4aa1a81df36db46bb225/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:03.888687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.918778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 24/43] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:03.941450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:03.988142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:04.012952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd141d8-d8be-4778-acce-4508e02d0f14 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:04.055473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:04.056969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:04.057262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:04.057583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:04.058821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:04.060873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:04.062371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 20/44] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:04.067714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:04.069295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be0cd3-8c95-4e58-9e67-9723ada54040 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:04.106098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:04.106838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:04.107102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:04.107378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:04.108144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:04.109561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928979b-bc92-4ac9-bf99-07c602fbcad7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:04.127478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 25/45] 10.222.0.65 () {60 vars in 1131 bytes} [Fri Jun 24 09:28:03 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:28:04.178621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee974-e491-49b7-b3ee-a2393df391b9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:04.217552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:04.218256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:04.218549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:04.218891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:04.219690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:04.221209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4cf9e-7cf2-41b5-90dd-01870fc68be7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:04.247518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:04.257535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 21/46] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:28:04 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:06.085347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9c4d-4363-496a-b817-b29c75c47cc0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.085855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9c4d-4363-496a-b817-b29c75c47cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.085855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af9c4d-4363-496a-b817-b29c75c47cc0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.087339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 26/47] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:06.092798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fde0d3-632f-4022-a6f1-7e9323b97443 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.093223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fde0d3-632f-4022-a6f1-7e9323b97443 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.093306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fde0d3-632f-4022-a6f1-7e9323b97443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.111215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91fde0d3-632f-4022-a6f1-7e9323b97443 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:06.112727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91fde0d3-632f-4022-a6f1-7e9323b97443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:06.293388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 22/48] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:06.316325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af9c4d-4363-496a-b817-b29c75c47cc0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.342883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:06.383966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104aa10-bc1b-460e-8988-bbedad4631a8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.384257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104aa10-bc1b-460e-8988-bbedad4631a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.384543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104aa10-bc1b-460e-8988-bbedad4631a8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.385936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 23/49] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:06.391339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d188-5672-435f-a016-90c571628c7c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.391629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d188-5672-435f-a016-90c571628c7c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.391908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d188-5672-435f-a016-90c571628c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.408595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5899d188-5672-435f-a016-90c571628c7c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:06.409793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5899d188-5672-435f-a016-90c571628c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:06.426570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af9c4d-4363-496a-b817-b29c75c47cc0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.463252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.464063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.464353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.464630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.474587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:06.481793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 24/50] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:06.482195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:06.483134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:06.483889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 27/51] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:06.489169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899d188-5672-435f-a016-90c571628c7c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.505784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.547660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:06.589417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:06.589784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899d188-5672-435f-a016-90c571628c7c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.604911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0927-d184-459b-a4b2-71cfd501acc1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.624008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.624692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.624983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.625284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.626397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:06.626709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:06.628192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:06.636334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 25/52] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:06.638927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.639780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.640068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.640347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.641529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:06.641876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.643013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:06.644674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 28/53] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:06.650213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.741410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461c4b-5a7f-4b2d-9e22-d2b4291b3fb5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.751951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050076f-0dc8-4d6c-8d0a-cb87ab4efb46 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.764894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:06.780387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.781369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.781724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.782075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.783355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:06.784847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:06.786513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 26/54] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:06.786648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.787292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.787579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.787940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.788654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:06.790116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:06.792135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.816791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 29/55] 10.222.0.65 () {60 vars in 1140 bytes} [Fri Jun 24 09:28:06 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:28:06.904093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca434e-40c6-43fe-b53f-74e3d1ed9848 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:06.944726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:06.945556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:06.945842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:06.946101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:06.947406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:06.948901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8161b-3a6c-4d4f-9220-f9d1ae3afaec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:06.950613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 27/56] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:06.956701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:06.961172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:07.060098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f59f8-a7af-489c-8455-47b446de978d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:07.095150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:07.095832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:07.096109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:07.096385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:07.097218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.domain.id=default, user.id=0fdc8f07afea4aa1a81df36db46bb225) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:07.097529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:07.098986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8660a8-51de-4ee4-9b7c-50b907abbc2c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:07.105041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 30/57] 10.222.0.65 () {56 vars in 1322 bytes} [Fri Jun 24 09:28:06 2022] GET /identity/v3/role_assignments?user.id=0fdc8f07afea4aa1a81df36db46bb225&scope.domain.id=default&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:08.893507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf770a-d57d-41e3-a950-3239b6f57ef6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:08.894045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf770a-d57d-41e3-a950-3239b6f57ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:08.894045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf770a-d57d-41e3-a950-3239b6f57ef6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:08.895492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 28/58] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:08.902195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:08.902846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:08.903129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:08.930325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:08.931573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:08.996272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:09.002862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 31/59] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:09.034636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf770a-d57d-41e3-a950-3239b6f57ef6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:09.094601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:09.146124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf770a-d57d-41e3-a950-3239b6f57ef6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:09.181741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073209-6491-4dee-b351-d087852758ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:09.182538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073209-6491-4dee-b351-d087852758ef None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:09.182823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073209-6491-4dee-b351-d087852758ef None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:09.183119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073209-6491-4dee-b351-d087852758ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:09.184441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073209-6491-4dee-b351-d087852758ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:09.185909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073209-6491-4dee-b351-d087852758ef None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:09.187577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 29/60] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:09.193868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:09.300243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cda8a7-dfb4-4789-ab3d-702ccc7126e2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:09.334336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:09.335001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:09.335314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:09.335589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:09.336273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:09.337741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aaf0f-ff62-4add-8d5a-ea1ad438a87f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:09.371921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 32/61] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:09 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:12.566796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586f38-88ea-4ca4-9ea3-3fc3eff5e0d7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.567873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586f38-88ea-4ca4-9ea3-3fc3eff5e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.568488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586f38-88ea-4ca4-9ea3-3fc3eff5e0d7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.570079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 30/62] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:12.575559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.575996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.576114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.608113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:12.609619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:12.644428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d93d-2996-491a-8886-60f70117c685 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.645065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d93d-2996-491a-8886-60f70117c685 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.645554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d93d-2996-491a-8886-60f70117c685 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.647519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 31/63] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:12.652686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225ae0-7e2a-42e9-a7ea-1e45b12d82bc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.653216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225ae0-7e2a-42e9-a7ea-1e45b12d82bc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.653690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225ae0-7e2a-42e9-a7ea-1e45b12d82bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.682625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bf225ae0-7e2a-42e9-a7ea-1e45b12d82bc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:12.683838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf225ae0-7e2a-42e9-a7ea-1e45b12d82bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:12.686481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:12.716802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 33/64] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:12.743846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:12.811496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 32/65] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:12.813138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2b8f-0a58-4584-a1c6-39a8dda6bbfc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.813433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2b8f-0a58-4584-a1c6-39a8dda6bbfc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.813713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee2b8f-0a58-4584-a1c6-39a8dda6bbfc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.815989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 33/66] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:12.821641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.822181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.822657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.840118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:12.841607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:12.883135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd9210-784b-4cc7-9029-afa4a6d97c2c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:12.895051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:12.916907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 34/67] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:12.923935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:12.929612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:12.929612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:12.929612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:12.929612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:12.942199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:12.952119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:12.953052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:12.953810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:12.956455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 34/68] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:12.958145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.010964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.045229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96f2a-9fe3-4e43-aab5-542260fd1d86 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.071854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22bd6-52c7-49a9-a69f-96ccde7fae9e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.083799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.084704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.085099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.085452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.097301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:13.105007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:13.105858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 35/69] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:13.111640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.113857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.115586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.115876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.116157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.128956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-eb3e7a1f-a7bf-4d63-ae7b-cf2df712a80e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:13.137147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:13.137999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:13.138766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 35/70] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:13.140064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5065-6a26-4588-a60a-d58408a6132a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.140064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5065-6a26-4588-a60a-d58408a6132a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.140178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5065-6a26-4588-a60a-d58408a6132a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.141410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 36/71] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:13.146148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd5065-6a26-4588-a60a-d58408a6132a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.164358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.219831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead870-5842-45c4-afbf-11f3730c5ecc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.248650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd5065-6a26-4588-a60a-d58408a6132a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.251303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.253634 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.254318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.254607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.254889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.255807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:13.256405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.257547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.265883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 36/72] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.266773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.282904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.283617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.283898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.284175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.284926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:13.285231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.286780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1bf12-a02a-403c-961c-6e55ab0351a5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.295233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 37/73] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.296532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923a6e-d561-4b9f-a8e1-10321fadafc0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.296823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923a6e-d561-4b9f-a8e1-10321fadafc0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.297100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923a6e-d561-4b9f-a8e1-10321fadafc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.313138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4b923a6e-d561-4b9f-a8e1-10321fadafc0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:13.314343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4b923a6e-d561-4b9f-a8e1-10321fadafc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:13.374108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4e93d-19fc-4c98-9d4f-8b28920f635e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.378922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 38/74] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:13.380314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca56db5-9d34-4cf8-b0e3-2eea096c8494 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.380664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca56db5-9d34-4cf8-b0e3-2eea096c8494 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.380943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca56db5-9d34-4cf8-b0e3-2eea096c8494 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.381663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.382356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 39/75] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:13.382944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca56db5-9d34-4cf8-b0e3-2eea096c8494 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.419773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.420453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.420738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.421011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.421757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:13.422074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.423500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.431729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 37/76] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.432537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.482205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.489210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca56db5-9d34-4cf8-b0e3-2eea096c8494 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.523364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.524258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.524561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.529077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.530222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429551ca-3fad-4c0e-aaa2-37275e3a4716 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.542289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.543839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf17da-5b19-49c6-88b1-d0cda8f05a9d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.545781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 40/77] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.546739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.547029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.547332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.563473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:13.564693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:13.569176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.570067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.570350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.570630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.571889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.573329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.574955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 38/78] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.575730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.610616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.633692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 41/79] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:13.634563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.679363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fbf53-195d-4a4f-87d8-640876d0ac66 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.712550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.713359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.713644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.713923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.724048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:13.731717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:13.732678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:13.733424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 39/80] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:13.734071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.734144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.752697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe65c5-d253-46c2-a818-a37fc1b49f45 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.800812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.801753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.802044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.802327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.803909 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.806033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:13.807663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 42/81] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:13.808448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:13.847921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.855310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6d79-4bb2-4639-81cf-efc2ee3a5018 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.898724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.899981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.900430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.900765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.918786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f852f-f300-4a4a-9a61-79f1819a2cff None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:13.957595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:13.958267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:13.958629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:13.958916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:13.978447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:13.980408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:13.980826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.033701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.035253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.082765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.094095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 40/82] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:14.099612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.141383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 43/83] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:14.143321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.198752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.224859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8bbd8-42ef-4dd0-a969-706e66536e33 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.246203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d4f12-af59-4193-9311-8f6511c554f5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.257902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.258741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.259032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.259346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.260526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.261989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.263667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 41/84] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.264436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.282872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.283603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.283891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.284170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.284927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:14.285235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.286688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.294940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 44/85] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.295776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.331041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.369893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc5344-63ee-4553-9225-7693b9a50b25 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.394184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa02226-de82-4956-9b14-9cc03fc0cf44 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.403612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.404277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.404566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.404909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.408648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.428875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.429860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.430134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.430417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.431655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.433148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.434889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 45/86] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.435682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.478276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.479771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.537940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795ccad-39e6-4f71-8362-8274f57818af None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.544165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.571335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.572216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.572499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.572765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.573971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.575469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.577163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 46/87] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.577952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.583821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 42/88] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:14.584480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.628605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.678384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20153752-caa2-4a6a-b31b-48dc920c8a1a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.685274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81324f4e-47bc-4f4b-bd29-67b563124fa4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.712046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.712732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.713261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.713261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.714004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.715513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.720589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.721485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.721773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.722058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.723299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.724822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.726481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 43/89] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.727367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.737582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 47/90] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:14 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:14.738344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.768868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.858440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f961-1861-4c0e-bec8-59bc1745ad43 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.867683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:14.873004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965eafb6-6f2c-44a4-9a12-71302810a314 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:14.904534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.905380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.905673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.905957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.906816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=0fdc8f07afea4aa1a81df36db46bb225) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:14.907182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.908672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.913835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:14.914517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:14.914802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:14.915206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:14.915957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:14.916271 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:14.917977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:14.919945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 48/91] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/role_assignments?user.id=0fdc8f07afea4aa1a81df36db46bb225&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.920843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:14.926574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 44/92] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=58709cf0655a474eb4a33606ebb525df => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:14.928564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:15.047692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:15.056095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722b6e5-a113-4e24-bb88-337befdb3a56 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:15.059299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100d7a-2f66-4cd9-a90b-60f0cc80c86b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:15.099436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:15.100437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:15.100715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:15.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:15.102296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:15.105003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:15.106771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:15.106920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 49/93] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:15.107934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:15.108315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:15.108604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:15.113365 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:15.163627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:15.205259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:15.207765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:15.252608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b737-3c32-47f1-a7b9-27395d0133cb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:15.300324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:15.300915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:15.301204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:15.301480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:15.302407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:15.302768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:15.304499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:15.311371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 50/94] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:15 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=561eaf0348e94e0f924cd58981968818 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:15.341019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:15.351410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 45/95] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:15.357219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:15.480465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214780c4-83c5-4999-bfce-62d66392d3c3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:15.482112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:15.526181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:15.527473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:15.527957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:15.528426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] PATH_INFO: `/v3/projects/51f92548c36f4a4dbe2d7735765eb115` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:15.529870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] RBAC: Authorizing `identity:get_project(project_id=51f92548c36f4a4dbe2d7735765eb115)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:15.531616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa712725-9adb-4b9a-98b1-b29887cb9a06 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:15.533859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 51/96] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:15 2022] GET /identity/v3/projects/51f92548c36f4a4dbe2d7735765eb115 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:15.540073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:15.687212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1178c-cb43-4fd2-85f1-a86a3cb63ba2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:15.733244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:15.733919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:15.734205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:15.734493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:15.735355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=51f92548c36f4a4dbe2d7735765eb115, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:15.735671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:15.737148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01720c5d-8403-47e6-8b66-ecdf6966ea84 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:15.747863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 46/97] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:15 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=51f92548c36f4a4dbe2d7735765eb115&role.id=58709cf0655a474eb4a33606ebb525df => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:17.158660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec6ed-b81e-4942-93e1-2197316cef32 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:17.159685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec6ed-b81e-4942-93e1-2197316cef32 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:17.160169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec6ed-b81e-4942-93e1-2197316cef32 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:17.162092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 52/98] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:17.167104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:17.167956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:17.168428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:17.189395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:17.190916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:17.249557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:17.262516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 47/99] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:17.285204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ec6ed-b81e-4942-93e1-2197316cef32 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:17.412688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ec6ed-b81e-4942-93e1-2197316cef32 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:17.458674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03807617-1c52-4710-b76f-f111447b102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:17.460110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03807617-1c52-4710-b76f-f111447b102e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:17.460401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03807617-1c52-4710-b76f-f111447b102e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:17.460681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03807617-1c52-4710-b76f-f111447b102e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:17.476130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03807617-1c52-4710-b76f-f111447b102e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-03807617-1c52-4710-b76f-f111447b102e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:17.484121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:17.484974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:17.485710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 53/100] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:17.492628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:17.628180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a388b53-400b-45ac-81b0-133cd7b8836c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:17.673271 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:17.673904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:17.674199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:17.674483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:17.675326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:17.675644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:17.677131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:17.686970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 48/101] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:17.692572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03807617-1c52-4710-b76f-f111447b102e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:17.725560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:17.821202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03807617-1c52-4710-b76f-f111447b102e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:17.871793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:17.872762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:17.873050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:17.873376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:17.874620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:17.876490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4ccad-8fb4-4a68-97db-c74d01821ead None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:17.878148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 54/102] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:17 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:17.883765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:17.955332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:18.006197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989d7c-cdf1-4edd-a1a3-cddc2c463d23 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.006866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989d7c-cdf1-4edd-a1a3-cddc2c463d23 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.007393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989d7c-cdf1-4edd-a1a3-cddc2c463d23 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.009021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 55/103] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:18.015137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a413b15-7d8a-4dd5-b8ee-30ec8da9dc94 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:18.020660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.021248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.021746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.052918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:18.054493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:18.055694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.056190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.057637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:18.058126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.059576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:18.061555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:18.063714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 49/104] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:17 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:18.072026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:18.172730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 56/105] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:18.212993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:18.242287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd3ea4-cb7b-45d9-82ac-070081459f62 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:18.262614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:18.289924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:18.320214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:18.321140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.321431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.321715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/0fdc8f07afea4aa1a81df36db46bb225/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.328468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, user_id=0fdc8f07afea4aa1a81df36db46bb225, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:18.330278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ae30b-8e23-4718-9e89-c818e94fd2b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:18.357335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 50/106] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:18 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/0fdc8f07afea4aa1a81df36db46bb225/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:18.359032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c031f58-e6c3-47a9-8575-dec20d185d21 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.360465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c031f58-e6c3-47a9-8575-dec20d185d21 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.360465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c031f58-e6c3-47a9-8575-dec20d185d21 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.361501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 51/107] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:18.381037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eb2b6-3fa0-492b-95c6-0159c693b776 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:18.383251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.383763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.384232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.412753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:18.414247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:18.434449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:18.435312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.435603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.435885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.438707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:18.440261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:18.442108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 57/108] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:18.448446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:18.567643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 52/109] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:18.612244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:18.649266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:18.662940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab351db-3379-4f43-8a87-0da58893261f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:18.730640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:18.731815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.732465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.732925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.733822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:18.735753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77c44d-4146-4f76-8151-2b862445a5a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:18.777472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b895c-dae9-483f-8a34-2c9ba8d4356d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:18.782190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 58/110] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:28:18 2022] POST /identity/v3/projects => generated 302 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:18.783805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dae0-4d02-44eb-a912-3f3cdd5510c8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.784917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dae0-4d02-44eb-a912-3f3cdd5510c8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.784917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dae0-4d02-44eb-a912-3f3cdd5510c8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.786075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 59/111] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:18.794010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9e3c-c3f4-4128-ae30-a97cffce5c33 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.794480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9e3c-c3f4-4128-ae30-a97cffce5c33 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.794948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9e3c-c3f4-4128-ae30-a97cffce5c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.797950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:18.822716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a0ca9e3c-c3f4-4128-ae30-a97cffce5c33 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:18.824713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a0ca9e3c-c3f4-4128-ae30-a97cffce5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:18.828358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:18.829480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.829975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.830460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.843852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:18.852901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:18.853845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:18.854619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:18.857542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 53/112] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:18.864385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:18.912197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 60/113] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:18.913762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026710-b756-4a1e-9a42-631d3b6cc351 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.914150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026710-b756-4a1e-9a42-631d3b6cc351 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.914527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58026710-b756-4a1e-9a42-631d3b6cc351 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.920522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 61/114] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:18.928833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:18.929456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:18.929948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:18.961097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:18.962694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:18.996277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88afe84-9b6d-4dc1-87df-e05cb661011e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.046331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.047018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.047335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.047613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.048438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:19.048746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.051110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e80c7-44e4-402c-bb5c-103c41466112 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.056752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 62/115] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:19.057964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.064808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 54/116] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:19.066455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdc12-6b3e-4831-a515-3f837e4558b6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.066958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdc12-6b3e-4831-a515-3f837e4558b6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.067459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdc12-6b3e-4831-a515-3f837e4558b6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.069253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 55/117] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:19.074458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.075058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.075607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.076081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.094882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:19.096489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:19.126930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.177774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 56/118] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:19.178967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.190852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d26e10-87a0-4109-8e46-57baa4d4d046 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.242381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.243631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.244150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.244627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.255888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:19.264020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:19.264928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:19.265685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:19.268081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 63/119] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:19.273628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.300179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1d3b-28db-4fb8-95db-01ce846d384e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.342142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.347909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.349161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.349835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.350474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.351965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.353701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.355681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 57/120] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:19.356648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.388218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f610-592d-4581-8a5d-bf9f252561cf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.390514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.429263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.430529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.431067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.431556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.442721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:19.451566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:19.452425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:19.453183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:19.455866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 64/121] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:19.456941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.479187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993cc02-3297-4c94-a7a5-eb8723846a4b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.527545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.528378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.528676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.528961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.542676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:19.553971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:19.554850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:19.555637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 58/122] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:19.556261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.576941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c44a1-8927-44cb-80d7-8b485acf44c3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.606389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.616154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.617279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.617769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.618235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.619229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:19.619734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.621424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.630892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 65/123] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:19.636092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.672489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.694113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef88cd3-de23-481b-9808-1b7900d00611 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.740072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.741512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.741808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.742095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.743731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.745986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.748630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 59/124] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:19.749954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.756494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9b1d5-1c79-446a-aad0-d9e8e714f537 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.793178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.794294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.794818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.795315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.796261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:19.796763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.798434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.807905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 66/125] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:19.809082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.874495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3caa43f-f486-44d6-b57e-0105b103069c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.878691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.912286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.912973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:19.913261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:19.913542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:19.914519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:19.914638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:19.916040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:19.924899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 60/126] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:19.925692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:19.946864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93047-00e0-47b4-88af-b4545b0ba8e3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:19.952779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:19.997783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:19.999594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.000098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.001191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] PATH_INFO: `/v3/users/3421a9cf90674d829e392ae5d02dd2d1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.022343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] RBAC: Authorizing `identity:get_user(user_id=3421a9cf90674d829e392ae5d02dd2d1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.024221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 67/127] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:19 2022] GET /identity/v3/users/3421a9cf90674d829e392ae5d02dd2d1 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.031141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.066484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431bd632-0a28-48eb-bb8b-317c902cf7ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.106598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.107704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.107997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.108283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/1de5aad186984d4aae5c980eb27c005a/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.111020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, user_id=1de5aad186984d4aae5c980eb27c005a, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.113466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.136168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 61/128] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:19 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/1de5aad186984d4aae5c980eb27c005a/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:20.139992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.170189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf147b-f58e-4f2f-b276-c2442ae69b8c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.189013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.243853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.245566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.245991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.246404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.248021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.250860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.256492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 68/129] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.259637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.324518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.367615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998991ea-75bd-4767-8b12-ceaf678a51ee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.382542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a178-8468-45ba-83da-0d935b7d44c0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.422724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.423982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.424410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.424851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.426277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.428129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.428399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.428688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.428968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.429611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.431784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 62/130] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.432785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.493421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.527568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.529753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.578003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7cc5e-8da0-4103-9781-5c798fed4504 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.647655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 69/131] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:20.649000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.660683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.662326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.662830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.663400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] PATH_INFO: `/v3/projects/51f92548c36f4a4dbe2d7735765eb115` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.664904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] RBAC: Authorizing `identity:get_project(project_id=51f92548c36f4a4dbe2d7735765eb115)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.666678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.675249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 63/132] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/projects/51f92548c36f4a4dbe2d7735765eb115 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.676483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.703864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.748634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:20.779818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a971b2-89d3-4864-a0d4-fada28f4cfbc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.816079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.817083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.817375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.817840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.819070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.820692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.822713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 70/133] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.823704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.841823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252b15-eed7-4010-a3bf-b554644c8d18 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.896996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.897901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.898190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.898469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.903805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.905298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:20.907371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 64/134] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:20 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:20.908136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:20.931822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af94ee-b6b1-4075-8535-ecbc3884bfee None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:20.972381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:20.973410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:20.973727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:20.974231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] PATH_INFO: `/v3/projects/51f92548c36f4a4dbe2d7735765eb115/users/1de5aad186984d4aae5c980eb27c005a/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:20.977637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=51f92548c36f4a4dbe2d7735765eb115, user_id=1de5aad186984d4aae5c980eb27c005a, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:20.979158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:21.011797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 71/135] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:20 2022] PUT /identity/v3/projects/51f92548c36f4a4dbe2d7735765eb115/users/1de5aad186984d4aae5c980eb27c005a/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:21.021733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:21.035419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:21.045317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:21.055326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b250fe7-5a2b-42d9-9147-834e08214996 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:21.106524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:21.107503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.107795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.108143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/1de5aad186984d4aae5c980eb27c005a/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.110581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, user_id=1de5aad186984d4aae5c980eb27c005a, role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:21.112192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b5ffe-8b31-4315-9780-c8429f67c15c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:21.139264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 65/136] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:20 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/1de5aad186984d4aae5c980eb27c005a/roles/561eaf0348e94e0f924cd58981968818 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:21.204297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dc73b-55b4-47a5-b235-d1c609333461 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.204591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dc73b-55b4-47a5-b235-d1c609333461 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.204844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dc73b-55b4-47a5-b235-d1c609333461 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.206571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 66/137] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:21.219700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750819d5-1a8a-4fcc-9de1-c354888aa6f4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.220033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750819d5-1a8a-4fcc-9de1-c354888aa6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.220289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750819d5-1a8a-4fcc-9de1-c354888aa6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.221495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e1afc-3d55-496c-b23a-948ded2a32a6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:21.260263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-750819d5-1a8a-4fcc-9de1-c354888aa6f4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:21.261898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-750819d5-1a8a-4fcc-9de1-c354888aa6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:21.262610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:21.263873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.264380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.264857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.265889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=3421a9cf90674d829e392ae5d02dd2d1) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:21.266409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:21.268121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:21.275498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 72/138] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:21 2022] GET /identity/v3/role_assignments?user.id=3421a9cf90674d829e392ae5d02dd2d1&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:21.369000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 67/139] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:21.404349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:21.515968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:21.528237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f28140-4273-4883-9e1e-668f45c65d03 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.529015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f28140-4273-4883-9e1e-668f45c65d03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.529437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f28140-4273-4883-9e1e-668f45c65d03 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.530976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 68/140] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:21.538806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89570d2a-0280-4624-b1b8-c7f6d829f52a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:21.540803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.541279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.541840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.580957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:21.582164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:21.587849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:21.588571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.588857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.589132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.616339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:21.632545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:21.633407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:21.634165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 73/141] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:21.642996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:21.761515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 69/142] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:21.822934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:21.828575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:21.842384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7b868-87e1-4970-8d02-7eb2324eaea7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:21.894364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:21.909570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:21.910652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:21.911153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:21.911834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:21.912817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:21.913343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:21.914989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:21.930036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 74/143] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:21.935976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:21.988187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d8b61-f20f-4a21-90b1-1008206df83c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:22.025110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:22.026316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:22.026793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:22.027295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:22.028698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:22.030438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:22.032458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 70/144] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:22.038232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:22.134417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:22.166116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038f4eb-d5ac-441d-851e-9d85945ded74 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:22.179051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65af1ac-fb78-4620-86fc-811bdd7098ef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:22.206663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:22.207731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:22.208211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:22.208792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:22.209707 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:22.211445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:22.243148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 71/145] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:22 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:22.261119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:22.262036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:22.262319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:22.262594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] PATH_INFO: `/v3/users/0fdc8f07afea4aa1a81df36db46bb225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:22.272277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] RBAC: Authorizing `identity:get_user(user_id=0fdc8f07afea4aa1a81df36db46bb225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:22.273848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:22.275987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 75/146] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:21 2022] GET /identity/v3/users/0fdc8f07afea4aa1a81df36db46bb225 => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:22.281624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:22.408722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08520c-9c10-47c8-9435-8d2f45fc1a74 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:22.439956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:22.454453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:22.455491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:22.458863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:22.459419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:22.538496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:22.540388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:22.673730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 72/147] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:22.678934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:22.784453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:22.855718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372418-f59f-440d-9dbe-8a31ebd5c3cb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:22.891905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:22.893209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:22.893728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:22.894190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:22.895601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:22.897614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e22b-ebdf-4dab-97ec-edb2482f423d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:22.899912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 76/148] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:22 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:22.905365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:22.938078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:23.031448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289261e4-aebe-440c-a2ab-7499b96ed887 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:23.071162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:23.072239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:23.072753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:23.073238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:23.074306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=0fdc8f07afea4aa1a81df36db46bb225) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:23.074819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:23.076533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5180c-cf4b-48f0-a323-fe1b6928f601 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:23.083712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 73/149] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:22 2022] GET /identity/v3/role_assignments?user.id=0fdc8f07afea4aa1a81df36db46bb225&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:23.526350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc949eb-b3bd-4778-b165-eb897ca14640 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:23.526861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc949eb-b3bd-4778-b165-eb897ca14640 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:23.526934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc949eb-b3bd-4778-b165-eb897ca14640 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:23.528514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 77/150] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:23.534311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:23.535234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:23.535729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:23.559004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:23.560583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:23.648655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 74/151] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:23.690838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc949eb-b3bd-4778-b165-eb897ca14640 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:23.841034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc949eb-b3bd-4778-b165-eb897ca14640 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:23.872549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:23.881502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:23.882484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:23.882775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:23.883197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:23.897714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:23.913247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:23.914151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:23.914915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 78/152] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:23.922147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:23.957104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:24.049183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be779954-1fa0-4c34-b53f-70269d2354b1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:24.121536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:24.122556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.123058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.123571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.124843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:24.125713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:24.128344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc24133-fb70-4475-aa5d-f5f6ef43555d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:24.144160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 75/153] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:24.153325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:24.209868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395729-2ebc-4d3b-ae3c-c7c7cab39f47 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.210700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395729-2ebc-4d3b-ae3c-c7c7cab39f47 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.211494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395729-2ebc-4d3b-ae3c-c7c7cab39f47 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.214104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 76/154] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:24.231227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.232092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.232791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.276449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:24.279998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:24.383878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c477-39b7-44fc-aa1c-f3df482765b8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:24.405965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 77/155] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:24.433669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:24.457345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:24.466726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:24.471140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.471140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.471140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.471140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:24.471140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:24.472860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 79/156] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:24 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:24.479550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:24.565714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e4e3c-c074-42e5-be18-6d1629f8a051 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:24.617637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:24.618233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:24.621222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.623244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.623685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.654216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d5c92fb6-237e-4a99-bcc2-7c3103fc502b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:24.674331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:24.675502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:24.676349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:24.683287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 78/157] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:24.686098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7402-d2e2-428f-aec5-f3fdb05b19f6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.686383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7402-d2e2-428f-aec5-f3fdb05b19f6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.686663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e7402-d2e2-428f-aec5-f3fdb05b19f6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.688079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 79/158] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:24.694793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.695130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.695358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.710902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391ef-c381-41ba-91ed-15e0550b12e5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:24.728148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:24.729442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:24.754893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:24.755920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.756459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.756878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.835727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:24.837605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e19ce-3714-4180-ba6d-4d9e1a8f3b2b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:24.897688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 80/159] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:24.898724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:24.902388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:24.912152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:24.962540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 80/160] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:24.964563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1232b4-97b4-4559-bfd4-aadda99f3457 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:24.965314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1232b4-97b4-4559-bfd4-aadda99f3457 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:24.965795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1232b4-97b4-4559-bfd4-aadda99f3457 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:24.967485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 81/161] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:24.968534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1232b4-97b4-4559-bfd4-aadda99f3457 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.076384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1232b4-97b4-4559-bfd4-aadda99f3457 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.113101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.114296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.114968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.115573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:25.136023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:25.136889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:25.137667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:25.140445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 82/162] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:25.141414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.183589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:25.188992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b8fe5-d678-432c-a187-4b7e785abdd8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.247092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc5f03-3aa3-44d6-b7d3-43e79e92e290 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.282391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.283669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.284157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.284620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.286050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:25.288032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd68f-ab13-4aca-af85-aae718e490c0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:25.289960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 83/163] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:25.291593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.292125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.292610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.310836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.311429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:25.311601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.311862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.312143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.312907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:25.313220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:25.313845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:25.314623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:25.345382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 81/164] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:25.346264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.387974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 84/165] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:25.389501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.427452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:25.492993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:25.501205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0e24-f4a6-42af-9381-69cc4da4de02 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.539764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.540847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.541348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.541836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.542865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:25.543408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:25.545254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:25.556583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 85/166] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=58709cf0655a474eb4a33606ebb525df => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:25.558362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.576975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5c960-041a-411d-8d09-39a62bfc27fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.668489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.669194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.669482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.669766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.670535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:25.670853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:25.673760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:25.693647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 82/167] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:25.694513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.707625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:25.751227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ce465-0257-4de9-86ee-feedae1aa9f4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.819511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.820435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.820720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.821002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.824053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:25.825665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:25.827550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 86/168] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:25.829422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:25.900959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd3ee5-24e6-4ba8-bfbf-8bd80deaabb1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:25.929116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:25.939134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:25.939134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:25.939134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:25.939134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:25.952257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:25.966380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:25.967430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:25.968243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 83/169] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:25.968883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.032328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5551-ec57-4366-8df1-73c17a4821a7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.086973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.088666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.088962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.089243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] PATH_INFO: `/v3/users/3421a9cf90674d829e392ae5d02dd2d1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.090796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=3421a9cf90674d829e392ae5d02dd2d1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.092346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4169-61f3-49ac-8fdc-869e6795e0a1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.094728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 87/170] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/users/3421a9cf90674d829e392ae5d02dd2d1 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:26.096587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898ece-845c-45fb-a509-bc449977d96e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.096874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898ece-845c-45fb-a509-bc449977d96e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.097273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898ece-845c-45fb-a509-bc449977d96e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.099733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 88/171] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:26.103572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898ece-845c-45fb-a509-bc449977d96e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.117888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280ea69-81e7-4b74-9ba3-8acd164d4f99 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.144173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.169154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.169827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.170102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.170443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.245704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.248723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898ece-845c-45fb-a509-bc449977d96e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.260511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.262131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b43df9-f4fc-47c3-a904-fe4e3fa09e5b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.298736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.299925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.300218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.300495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.301262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:26.301573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.303474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.312370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 89/172] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:26.317888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.388255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 84/173] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:26.390306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.390733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.391170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.406979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.420160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:26.422049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:26.434595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a21a-9b11-450e-aa0b-76c9e2a0cf05 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.472214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.474028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.474696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.475277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.476658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.478290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.480266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 90/174] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:26.481231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.507585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 85/175] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:26.514883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.546184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.612838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3efdd-68f3-4b32-b3c6-7edf360cf764 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.650887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.652084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.652524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.652943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] PATH_INFO: `/v3/users/1de5aad186984d4aae5c980eb27c005a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.653438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea45e99-fc6b-410c-a83d-5b532d391313 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.654901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] RBAC: Authorizing `identity:get_user(user_id=1de5aad186984d4aae5c980eb27c005a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.656675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.658616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 91/176] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/users/1de5aad186984d4aae5c980eb27c005a => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:26.659608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.676790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.693852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.694838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.695151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.695433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] PATH_INFO: `/v3/projects/51f92548c36f4a4dbe2d7735765eb115` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.696685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] RBAC: Authorizing `identity:get_project(project_id=51f92548c36f4a4dbe2d7735765eb115)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.698450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.705126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 86/177] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/projects/51f92548c36f4a4dbe2d7735765eb115 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:26.705827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.773378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee6b22-5363-4386-994c-f12f0efaf0fa None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.795058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.808052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.808958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.809236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.809509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/3421a9cf90674d829e392ae5d02dd2d1/roles/b16d427c48574ab1b79bcd0f1f571c27` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.810224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60f4-7794-4a5f-a599-0ea420046da2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.811542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, user_id=3421a9cf90674d829e392ae5d02dd2d1, role_id=b16d427c48574ab1b79bcd0f1f571c27)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:26.813019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:26.826334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 92/178] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:26 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/users/3421a9cf90674d829e392ae5d02dd2d1/roles/b16d427c48574ab1b79bcd0f1f571c27 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:26.839340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.845043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:26.845895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.846179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.846458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.859051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-21d0c320-7543-4b5b-bba0-aeda8b3ddb02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:26.867879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:26.869000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:26.869970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 87/179] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:26.870754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86457c-7cd1-4c1c-88c1-3ddce72c5138 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:26.870754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86457c-7cd1-4c1c-88c1-3ddce72c5138 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:26.870985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86457c-7cd1-4c1c-88c1-3ddce72c5138 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:26.872421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 88/180] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:26.873137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86457c-7cd1-4c1c-88c1-3ddce72c5138 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:26.928248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:26.968041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bfaa-9059-4203-ba3c-68303c5293a9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:26.994332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86457c-7cd1-4c1c-88c1-3ddce72c5138 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.013142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.013814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.014097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.014372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.034027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.034804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.035118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.035401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.036508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=51f92548c36f4a4dbe2d7735765eb115, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:27.036614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.038222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656636a7-3e5a-49b7-a4c9-f8bb5ad986c0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.044473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 89/181] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=51f92548c36f4a4dbe2d7735765eb115&role.id=58709cf0655a474eb4a33606ebb525df => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.045502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f593f-a626-4563-955b-c07db29672e7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.045795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f593f-a626-4563-955b-c07db29672e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.046068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f593f-a626-4563-955b-c07db29672e7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.047339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 90/182] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:27.048383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.048671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.048950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.077038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:27.078439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:27.083724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.117641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.120495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b273d6-3903-424a-980a-3004972c75b0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.167716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 91/183] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:27.171708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.224567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.262135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 93/184] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:27.264749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.265041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.265318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.285420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:27.287063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:27.313251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72896eac-3c5b-4031-a1b2-9a13936fe76a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.349584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.350264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.350547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.350823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.351617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:27.351918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.353312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.366148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 92/185] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.366148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.366148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.372851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 94/186] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:27.375924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.474374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366227-77a8-4453-97cd-570dfc5d392d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.476976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7380e-88ec-456b-9262-233b95d8e67e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.478550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.510216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.511037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.511348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.511624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.516081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.517127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.517736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.518171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.519816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.521521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.522562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:27.523582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 95/187] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.524471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:27.530593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:27.531619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:27.532432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 93/188] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:27.533068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.611048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.630682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b600970-43eb-41b9-bb48-b82a0c630c22 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.655537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8805a0-8dba-4622-8a09-3e7a1e98331f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.666817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.667731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.668011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.668278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.682720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.684637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.690704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 96/189] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.690704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.696252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.697071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.697361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.697645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.698821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.700740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.703246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 94/190] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.708138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.722146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.801243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2c32c-f44e-48bb-8c1c-1555ef5faef1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.817931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a87f4-7276-4259-9a70-bcb006d5848f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:27.836062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.836731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.837011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.837290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.838562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=1de5aad186984d4aae5c980eb27c005a) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:27.838874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.840369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.855124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 97/191] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/role_assignments?user.id=1de5aad186984d4aae5c980eb27c005a&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=561eaf0348e94e0f924cd58981968818 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.855124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.856609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.866709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:27.867553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:27.867904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:27.868559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:27.869088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:27.869462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:27.870949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:27.883802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 95/192] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:27.884756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:27.965390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:27.976580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140679b-486c-4d91-9b6a-a82fdf55a169 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.020081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c27c-fa29-415f-b140-ab125a275abf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.021164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.021860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.022452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.022741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.058391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.059276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.059644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.059923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.060747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.062355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.089627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 96/193] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:27 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:28.090479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.112242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.113833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.172727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.223871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.233744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 98/194] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:28.235944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fb221-9e2a-4f0c-a57a-43aa6eee586d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.239542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.277068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.278379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.278855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.279354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.280982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.282675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.286972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 97/195] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:28.290648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.353336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cba18d-4f26-457e-b739-c80f5157bf18 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.400477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.401528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.401873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.402484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.403991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.406240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.408692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 99/196] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:28.414744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.416895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392775a-25c2-4c49-8283-01ccabfcc7a6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.438200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.463339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.464072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.464366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.464753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.484135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.529372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6c0c1-3479-4415-b94a-45e8fb055de0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.545672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.547521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.564494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.565170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.565458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.565733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.566567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=ee128c79d7a44d16ae0294098ceffb50) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:28.566876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.568422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.574611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 100/197] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/role_assignments?user.id=ee128c79d7a44d16ae0294098ceffb50&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:28.675366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 98/198] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:28.681896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.819649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92369c1-e400-44b2-990c-7e484ecf7110 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:28.853115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.857456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:28.858426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.858772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.859142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.860567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:28.862291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e288a5-25fb-4879-99f8-9289aafe802f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:28.864692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 101/199] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:28.870418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:28.893292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:28.944749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73b70d-d9d5-4002-844e-8b6998d9be49 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.945052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73b70d-d9d5-4002-844e-8b6998d9be49 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.945306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73b70d-d9d5-4002-844e-8b6998d9be49 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.948004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 102/200] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:28.952142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:28.952429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:28.952713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:28.976172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:28.977391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:29.026182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561bd28-826e-4e69-9319-cc2266400705 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:29.088675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:29.089355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:29.089641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:29.090124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:29.092302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=ee128c79d7a44d16ae0294098ceffb50) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:29.092843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:29.094330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:29.107134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 99/201] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:28 2022] GET /identity/v3/role_assignments?user.id=ee128c79d7a44d16ae0294098ceffb50&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=561eaf0348e94e0f924cd58981968818 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:29.112746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 103/202] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:29.136842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:29.228340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:29.249787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9ed4b-2e8f-43d9-8f24-55e0d96fa6d6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:29.286457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:29.287278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:29.287567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:29.287850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:29.307442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:29.315881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:29.316821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:29.319131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 100/203] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:29.323608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:29.451611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db9a7-f24a-417a-9e4c-7410c769af95 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:29.476972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:29.490261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:29.491092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:29.491384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:29.491661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:29.492488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:29.492804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:29.494233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:29.503982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 104/204] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:29.512270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:29.685235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c34760-342e-4c95-bd4f-07f590eb39e5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:29.734054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:29.735225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:29.735650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:29.736063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] PATH_INFO: `/v3/users/3421a9cf90674d829e392ae5d02dd2d1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:29.739800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] RBAC: Authorizing `identity:get_user(user_id=3421a9cf90674d829e392ae5d02dd2d1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:29.741432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:29.743303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 101/205] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:29 2022] GET /identity/v3/users/3421a9cf90674d829e392ae5d02dd2d1 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:29.749178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:29.898494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a7062-10d3-4f75-b67b-f1b321d5b527 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:29.946123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:29.946975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:29.947435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:29.947852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.011467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:30.024551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:30.026109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:30.124397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 105/206] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:30.131032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:30.144898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:30.256895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ba069-2a57-44ba-ad44-fbb43b64a87e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:30.299720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:30.300636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.300924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.301204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.303239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:30.304749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393176aa-c0ec-42f8-9130-39a915a58af4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:30.311261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 102/207] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:30 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:30.313285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:30.401775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:30.417809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddac2e7-bd7c-4c26-b51b-77e5b942c13e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:30.454107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:30.454787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.455222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.455543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.456452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, scope.project.id=628cdb8129ea4f1191d369f8afd5294c, user.id=3421a9cf90674d829e392ae5d02dd2d1) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:30.456910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:30.458840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa410550-25a0-41cd-ad2e-3a50d2dcf1a6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:30.465834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 106/208] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:30 2022] GET /identity/v3/role_assignments?user.id=3421a9cf90674d829e392ae5d02dd2d1&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:30.905741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf266-aaf5-4394-8470-ee40b95b7096 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.906245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf266-aaf5-4394-8470-ee40b95b7096 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.906318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcf266-aaf5-4394-8470-ee40b95b7096 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.907993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 103/209] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:30.913741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.914192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.914267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.917819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee3da-3a67-4b8a-a00a-109a88618f08 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.918210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee3da-3a67-4b8a-a00a-109a88618f08 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.918478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee3da-3a67-4b8a-a00a-109a88618f08 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.920210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 104/210] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:30.935647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:30.936441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:30.936825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:30.936937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:30.937270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:30.967434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:30.968790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:31.027460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 107/211] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:31.053253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.058464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 105/212] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:31.090588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.100252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.141342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.165734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854d76e-6980-42b2-a86a-eb4f0c9efa12 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.200114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.201279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.201783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.202250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.203710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:31.205390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:31.207483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 108/213] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:31.209275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62df341-992e-472c-9600-976bde0e0d7b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.213687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.247908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.248982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.249577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.250390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.262554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:31.271425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:31.272413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:31.273209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:31.275302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 106/214] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:31.281394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.322949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bdd60-51a9-4360-b0ef-01d3b11c861f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.363155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abedae-de89-4178-bac3-b15580764595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.363923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abedae-de89-4178-bac3-b15580764595 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.364232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abedae-de89-4178-bac3-b15580764595 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.364587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abedae-de89-4178-bac3-b15580764595 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.365321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abedae-de89-4178-bac3-b15580764595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:31.366821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abedae-de89-4178-bac3-b15580764595 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:31.383345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.403105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 109/215] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:31 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:31.403105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f79b-8a1f-480d-8a05-4d238512f5c4 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.403105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f79b-8a1f-480d-8a05-4d238512f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.403105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f79b-8a1f-480d-8a05-4d238512f5c4 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.403105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 110/216] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:31.409775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.410225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.410602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.416562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.436299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:31.438184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:31.444789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88d5b2-6f90-46b6-bf37-973aa9aa5075 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.493012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.493983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.494413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.494832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.495795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:31.496394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:31.498813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:31.512669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 107/217] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:31.518902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.533232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 111/218] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:31.556565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.649076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd75e2-8a63-4f3b-88c9-0e1bc84da0f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.682130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.684686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714896e-6d6d-475e-a872-2f784047cac9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.696765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.697679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.697962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.698623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] PATH_INFO: `/v3/users/c7881256809045a580c808ce222e1077` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.721460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.727185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] RBAC: Authorizing `identity:get_user(user_id=c7881256809045a580c808ce222e1077)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:31.727675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31140d8-be42-4a76-a55a-f722f99cbe07 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:31.730434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 108/219] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/users/c7881256809045a580c808ce222e1077 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:31.732116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.732321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ad38-2ae8-44e2-806f-bf935aa160b3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.732571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ad38-2ae8-44e2-806f-bf935aa160b3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.732860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77ad38-2ae8-44e2-806f-bf935aa160b3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.733745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.734164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 109/220] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:31.734415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.734876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.734974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ad38-2ae8-44e2-806f-bf935aa160b3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.752816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f7668236-23cf-451c-b1e8-9ceae02d8a16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:31.760456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:31.761343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:31.762102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:31.762704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 112/221] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:31.763783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.764075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.764353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.789283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:31.790593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:31.852059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77ad38-2ae8-44e2-806f-bf935aa160b3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:31.866461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 113/222] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:31.867360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:31.914175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:31.915595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:31.915887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:31.919357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:31.989715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:31.992582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edac0a-abab-47e5-b9a3-9931e3a81b6c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.004581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.006239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.017644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.032046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.032949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.033367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.033776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.034689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:32.035177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.037231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.046764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 114/223] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.047945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.110818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 110/224] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:32.111740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.153056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdc51-4718-45b0-bb93-f9af2862f5f4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.188818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.190336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.190817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.191296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.200087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.205580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:32.216645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:32.217529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:32.218290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:32.218878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed98d6-1cdd-47eb-8f45-4bbc3e0597b6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.221408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 115/225] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:32.222415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.271282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.271282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.271282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.271282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.271713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.273240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.275311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 111/226] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.276236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.300325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.346957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee059d5-43a5-4c23-bb89-000509c91591 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.383481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.384589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.384928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.385265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.386563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.388434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.390664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 116/227] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.391329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.394966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee0bfb-30f5-4b19-a946-e20348eb012a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.432207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.433173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.433658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.434099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.435141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:32.435631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.437300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.446984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 112/228] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.448274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.470210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.503362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b81f-3785-45f9-907b-06823df41aba None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.517675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.540303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.541274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.541644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.541995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.543504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.545231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.547391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 117/229] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.548021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.559876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61491-9876-420c-8b03-ee5997333f9b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.595338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.596327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.596935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.597403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.598617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=c7881256809045a580c808ce222e1077) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:32.599146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.600907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.607283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 113/230] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/role_assignments?user.id=c7881256809045a580c808ce222e1077&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=58709cf0655a474eb4a33606ebb525df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.608424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.657560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b245-e848-48b1-9f2c-06f84376f4d1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.695715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.696676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.696956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.697287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.698535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.700055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.701125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.703331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 118/231] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.708642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:32.726325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e117a-89b4-4d3c-a114-f38f4c38c579 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.778533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:32.794236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.795549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.795968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.796378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/ee128c79d7a44d16ae0294098ceffb50/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.802762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, user_id=ee128c79d7a44d16ae0294098ceffb50, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.804562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.838173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc1d68-8a41-4cb9-9244-d931ac88979a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:32.839023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 114/232] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:32 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/ee128c79d7a44d16ae0294098ceffb50/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:32.877218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:32.878201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:32.878490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:32.878821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:32.880361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:32.881948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ccd59-32fd-4760-98f6-dbb5206ae9d4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:32.884112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 119/233] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:32 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:32.889540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:33.009934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc117af8-1a4d-4af8-a679-8caa2f81ea4f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:33.050805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:33.051847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:33.052217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:33.052779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/ee128c79d7a44d16ae0294098ceffb50/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:33.054844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, user_id=ee128c79d7a44d16ae0294098ceffb50, role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:33.056883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52681d-82e2-44a6-9003-7cf6cebc2f4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:33.072304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:33.078136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 115/234] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:32 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/ee128c79d7a44d16ae0294098ceffb50/roles/561eaf0348e94e0f924cd58981968818 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:28:33.925257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268511a-4cac-46e1-b7d3-e842390b0340 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:33.925769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268511a-4cac-46e1-b7d3-e842390b0340 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:33.925874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268511a-4cac-46e1-b7d3-e842390b0340 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:33.927526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 120/235] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:33.928173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa535-a89a-4caf-8c02-42a74bd2363d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:33.928671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa535-a89a-4caf-8c02-42a74bd2363d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:33.929131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaa535-a89a-4caf-8c02-42a74bd2363d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:33.930694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 116/236] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:33.939337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:33.939453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:33.939637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:33.939910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:33.940293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:33.940751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:33.959282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:33.960798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:33.964076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:33.966219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:34.048649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 117/237] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:34.074343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.097728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 121/238] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:34.150541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.187597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c61aab-5cc4-47ba-b6a3-86ebadd116b6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:34.220570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:34.227801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:34.229001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:34.229432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:34.230022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:34.241863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:34.250874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:34.251987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:34.252969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:34.255505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 118/239] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:34.262338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.290200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:34.373884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243fa06-cfa9-40f0-8fd2-e0fadef75fd2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:34.410927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:34.412130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:34.412560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:34.412979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:34.413875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:34.414324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:34.416056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:34.425788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 119/240] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:34.432198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.490337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:34.505899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19168563-de71-4f8e-804a-44fafc74e969 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:34.542263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173a1e-de71-47dd-a74b-30b31442381a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:34.580062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:34.581221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:34.581648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:34.582065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] PATH_INFO: `/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:34.598256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] RBAC: Authorizing `identity:get_user(user_id=48a7445d5f874ed6815db1a8a0ce5fa1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:34.600259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:34.602854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 120/241] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:34.608995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.629248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:34.630062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:34.630343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:34.630637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:34.631970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:34.633819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:34.635170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 122/242] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:34.640939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.722887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2694-3bc0-4bd4-b046-6f96d951ab10 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:34.765400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:34.773173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:34.774386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:34.774844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:34.775330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:34.852554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:34.854346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:34.971850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 121/243] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:34.978517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:34.991342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.009846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95962c-08e3-4645-a24e-dcd77eb315fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.044530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.094273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19a299-b7a8-4b1f-80bf-9b7722deefb6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.111815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.112577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.112809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.113113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.113824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:35.115617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:35.136026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.137192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.137647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.138063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.139448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:35.141046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a26ce2-7e0d-4064-8eaa-d13a29b6fc89 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:35.142970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 122/244] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:34 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:35.143810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 123/245] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:34 2022] POST /identity/v3/users => generated 293 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:35.144504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f81f-dd7e-4d91-b111-22efe76f238c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.144845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f81f-dd7e-4d91-b111-22efe76f238c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.145124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f81f-dd7e-4d91-b111-22efe76f238c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.146683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 123/246] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:35.151939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.152604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.152993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.153368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.171432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:35.172879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:35.250579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 124/247] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:35.274579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.287507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.348087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6e1ab-e29d-43d8-8e1b-04efdb662eb7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.390061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70f483-e312-4110-890f-ab044ce191c3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.402699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.403432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.403721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.403993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.404818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=48a7445d5f874ed6815db1a8a0ce5fa1) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:35.405127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:35.406712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a7a-1930-441d-b1e7-f0e7c2f09112 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:35.418090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 124/248] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/role_assignments?user.id=48a7445d5f874ed6815db1a8a0ce5fa1&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:35.433147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.434437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.434875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.435327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.439390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e52b05-2d51-4ee1-b64c-089516b98962 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.439812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e52b05-2d51-4ee1-b64c-089516b98962 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.440181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e52b05-2d51-4ee1-b64c-089516b98962 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.441752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 125/249] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:35.448347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.448790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.449068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.452523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:35.468076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:35.469049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:35.469831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:35.473138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 125/250] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:35.477838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:35.479646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:35.480029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.513294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.572409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 126/251] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:35.577266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.606135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.619508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d73b12-0880-4c01-8086-164b09febd94 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.675347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.676140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.676540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.677046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.677865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:35.678183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:35.679892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:35.689514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 126/252] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:35.694635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.725584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568b501-8d35-49a0-9083-cb31aafbce55 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.769480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.770283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.770605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.770880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.782838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-510d9b9b-eff0-4c57-bd6d-4354bdf05edb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:35.791619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:35.792552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:35.793348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 127/253] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:35.794495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758b09-ba52-48c5-991c-8e99e5e39497 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.794495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758b09-ba52-48c5-991c-8e99e5e39497 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.794716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758b09-ba52-48c5-991c-8e99e5e39497 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.796262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 128/254] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:35.803193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.803570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.803851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.829033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:35.830529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:35.837162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8851b57-eab4-42da-b57d-51fe6ec2ad24 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:35.853820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.871772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:35.879731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:35.880993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:35.881498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:35.881971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] PATH_INFO: `/v3/users/c7881256809045a580c808ce222e1077` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:35.883462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] RBAC: Authorizing `identity:get_user(user_id=c7881256809045a580c808ce222e1077)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:35.885281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:35.887615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 127/255] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/users/c7881256809045a580c808ce222e1077 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:35.888766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:35.917427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 129/256] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:35.918120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.037547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf48b-67c1-42b1-9bdb-9684f46a2b9c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.049238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275638b4-1a77-418f-b5ea-6254f5e9a698 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.077448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.078139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.078427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.078706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.079994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:36.080312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.081884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:36.092075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 128/257] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:36.092924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.135821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.137498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.137781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.138073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.143647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.145202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:36.147223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 130/258] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:35 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:36.149666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.180395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:36.206797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:36.268708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2542-8c6a-4ca0-a1a9-3e00e2d9ed74 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.311971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.312808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.313098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.313380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.316147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.317715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:36.319436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 131/259] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:36 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:36.320189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.322597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d786e7-a411-4a28-b9db-580b21f97ded None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.427491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:36.436328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1021c-8908-4e49-ab92-a50b119b630b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.460375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.461263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.461527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.461876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.475284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.476640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.477114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.477579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/c7881256809045a580c808ce222e1077/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.479772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, user_id=c7881256809045a580c808ce222e1077, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.481467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:36.488078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:36.497440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:36.498418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:36.499257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:36.507929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 132/260] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:36 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/c7881256809045a580c808ce222e1077/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:36.509049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.515355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 129/261] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:36.520782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.717993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fbfb8-ced4-4cf2-b75d-ad0190bc6d63 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.732278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745307-e7f9-428c-a9e0-859ba42e9341 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:36.755531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:36.787845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.788549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.788835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.789112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.789881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:36.790192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.791653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:36.809840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:36.810514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:36.810794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:36.811066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:36.813215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 130/262] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:36.819026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:36.832122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:36.894337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:36.896560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.005156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 133/263] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:37.011977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:37.057411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3028026-7bfe-4dbf-83ca-7fdfd14daccc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:37.064905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:37.126451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:37.127376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:37.127660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:37.127932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] PATH_INFO: `/v3/users/ee128c79d7a44d16ae0294098ceffb50` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:37.132476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] RBAC: Authorizing `identity:get_user(user_id=ee128c79d7a44d16ae0294098ceffb50)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:37.135986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0b27a-cd82-408b-bad0-c2d25fd276f8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:37.138376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.140299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 131/264] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:36 2022] GET /identity/v3/users/ee128c79d7a44d16ae0294098ceffb50 => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:37.145877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:37.205315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:37.206189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:37.206468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:37.206740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:37.208841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:37.212643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.214376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 134/265] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:37 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:37.225560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:37.260280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:37.335697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73dbb2-a535-4118-aa1b-91b13fb4a166 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:37.354180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:37.373906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:37.374582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:37.375289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:37.375561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:37.376438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=ee128c79d7a44d16ae0294098ceffb50) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:37.376749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:37.378215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.386762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862d18-2a88-4f43-ba50-36a7feaf4cbe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:37.388206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 135/266] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:37 2022] GET /identity/v3/role_assignments?user.id=ee128c79d7a44d16ae0294098ceffb50&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:37.445725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:37.446420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:37.446711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:37.446988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:37.539447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:37.540907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.598182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:37.657493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 132/267] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:28:37.663260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:37.813480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496bd1c-8d28-4db8-895b-9357024c62b3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:37.858210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:37.859121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:37.859406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:37.859675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:37.860898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:37.862492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d9458-49ae-4225-a1f5-ae5f26f6d0cc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:37.864507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 136/268] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:37 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:37.870607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.002117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78b65d-3b1f-4ec9-bee6-9887624cf7ff None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.026986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701be9-d5d8-41f3-a963-b02d17265f0b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.027322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701be9-d5d8-41f3-a963-b02d17265f0b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.027562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701be9-d5d8-41f3-a963-b02d17265f0b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.029062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 137/269] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:38.034249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b95c74-bfac-4726-9425-385aff6f2e45 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.034638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b95c74-bfac-4726-9425-385aff6f2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.034926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b95c74-bfac-4726-9425-385aff6f2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.052403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.053090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.053379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.053651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.054493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=ee128c79d7a44d16ae0294098ceffb50) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:38.054802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:38.056335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:38.067431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 133/270] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:37 2022] GET /identity/v3/role_assignments?user.id=ee128c79d7a44d16ae0294098ceffb50&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=561eaf0348e94e0f924cd58981968818 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:38.068475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d4b95c74-bfac-4726-9425-385aff6f2e45 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:38.069686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d4b95c74-bfac-4726-9425-385aff6f2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:38.158633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.190199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 138/271] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:38.228697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.246911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.340790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba815c-9976-41d0-b27e-af59e1560e50 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.382177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.383408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.383883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.384342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.401670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:38.408988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588651-d1cd-4fa4-b007-d51f4c5eeb83 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.409302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588651-d1cd-4fa4-b007-d51f4c5eeb83 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.409547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a588651-d1cd-4fa4-b007-d51f4c5eeb83 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.411105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 139/272] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cf939e50-1258-410b-9c91-47e54bc5295c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:38.412142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:38.413311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:38.414277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:38.416591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 134/273] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:38.416890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562922-1661-43cf-a201-0a14135d5004 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.417189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562922-1661-43cf-a201-0a14135d5004 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.417466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562922-1661-43cf-a201-0a14135d5004 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.418172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2626c8-23b3-4982-9e44-0388394a8312 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.418459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2626c8-23b3-4982-9e44-0388394a8312 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.418740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2626c8-23b3-4982-9e44-0388394a8312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.419138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 140/274] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:38.425276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562922-1661-43cf-a201-0a14135d5004 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.444305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dd2626c8-23b3-4982-9e44-0388394a8312 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:38.446701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dd2626c8-23b3-4982-9e44-0388394a8312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:38.515380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 135/275] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:38.524658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.524983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.525268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.532340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.547061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:38.548515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:38.549023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562922-1661-43cf-a201-0a14135d5004 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.585480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.586166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.586457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.586723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.587539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:38.587848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:38.589261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:38.598027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 141/276] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:38.598842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.625934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 136/277] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:38.625934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.653164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.712918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9effa9d-a048-4f1d-8276-d82d4b03ed1e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.736111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655240-29e7-4ff0-8d81-cd8ce200e68b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.750710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.751588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.751870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.752143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.766441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:38.774533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:38.775638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:38.776461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 142/278] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:38.777116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.781251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.782303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.782722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.783185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] PATH_INFO: `/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.784696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] RBAC: Authorizing `identity:get_user(user_id=48a7445d5f874ed6815db1a8a0ce5fa1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:38.786402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:38.791795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 137/279] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:38.793274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.796510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.886542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761267d-2e18-4b6f-8b39-202b30e7e730 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.888178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:38.899556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272924b6-5fd7-40b1-b8df-5267d1072674 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:38.923664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.924473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.924760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.925041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.926214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:38.927946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:38.929776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 143/280] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:38.932000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:38.934989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:38.935711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:38.936000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:38.936282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:38.937040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:38.937349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:38.938749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:38.951664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 138/281] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:38.951664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.017897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:39.039881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec5aa-3b22-4179-9c03-0d30ceea4950 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.058795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fdb9b-7464-4b36-bfee-3ce581923071 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.076284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.077761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.078288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.078726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.080097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.081944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.084811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 144/282] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:38 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:39.085846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.096179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.096870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.097174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.097444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.098161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.099726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.125230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 139/283] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:28:38 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:39.126050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.147280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:39.228994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cc060-e4e2-41b7-8b7f-c7c33b826bf6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.273382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47f85d-3c25-48a0-a86a-09f2c88c03b1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.280342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.281497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.281933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.282378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] PATH_INFO: `/v3/users/4c0c872427b540f3a3803e7a888535e8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.302190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] RBAC: Authorizing `identity:get_user(user_id=4c0c872427b540f3a3803e7a888535e8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.304811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.306379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:39.308132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 145/284] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:39 2022] GET /identity/v3/users/4c0c872427b540f3a3803e7a888535e8 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:39.315136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.329189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.330413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.330879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.331196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/48a7445d5f874ed6815db1a8a0ce5fa1/roles/b16d427c48574ab1b79bcd0f1f571c27` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.333264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, user_id=48a7445d5f874ed6815db1a8a0ce5fa1, role_id=b16d427c48574ab1b79bcd0f1f571c27)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.334769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d49a4-12f3-43c5-a8ac-0a4a4cefab33 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.351062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 140/285] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:28:39 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/users/48a7445d5f874ed6815db1a8a0ce5fa1/roles/b16d427c48574ab1b79bcd0f1f571c27 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:39.352309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa95-a921-4f1f-b497-62c7b00c2842 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.352605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa95-a921-4f1f-b497-62c7b00c2842 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.352943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa95-a921-4f1f-b497-62c7b00c2842 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.354388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 141/286] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:39.360534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.360843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.361100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.379502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:39.380886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:39.456779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:39.469106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b584c-9b4e-42a7-acec-87a132c042f8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.486739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 142/287] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:39.511474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.512167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.512451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.512733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.513710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, user.id=4c0c872427b540f3a3803e7a888535e8, scope.system=all) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:39.514035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.515679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.516748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.522840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 146/288] 10.222.0.65 () {56 vars in 1308 bytes} [Fri Jun 24 09:28:39 2022] GET /identity/v3/role_assignments?user.id=4c0c872427b540f3a3803e7a888535e8&scope.system=all&role.id=58709cf0655a474eb4a33606ebb525df => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:39.638256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ac26-cfcc-4fa0-8026-89c94ddfecfb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.652103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:39.680050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.680893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.681179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.681458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.693777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:39.701830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:39.702768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:39.703593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 143/289] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:39.709583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:39.854433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a3080-18fd-4355-8d38-52564bf2e18e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:39.901676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:39.902372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:39.902657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:39.902936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:39.903752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:39.904066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:39.905495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:39.918330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 147/290] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:39.923947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:40.060064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5ff33-7c87-47e9-8ed2-7a370aca8599 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:40.095110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:40.096011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:40.096292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:40.096570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] PATH_INFO: `/v3/users/c7881256809045a580c808ce222e1077` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:40.097797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] RBAC: Authorizing `identity:get_user(user_id=c7881256809045a580c808ce222e1077)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:40.099381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:40.101341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 144/291] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:39 2022] GET /identity/v3/users/c7881256809045a580c808ce222e1077 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:40.106876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:40.134232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:40.215297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583e996-7138-44d3-9c68-617e9ce255e5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:40.251782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:40.252512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:40.252800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:40.253128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:40.325554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:40.326961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:40.351770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:40.429721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 148/292] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:40.435300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:40.495198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:40.575251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420256a-a0db-49e8-89c6-e4e2bd55a014 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:40.619258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:40.620075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:40.620363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:40.620641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:40.621833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:40.623340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0345d3-e064-43eb-9702-f4b08f93a768 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:40.625437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 145/293] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:40 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:40.631384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:40.738575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439cae7-a327-4004-bdf5-8d675cafd774 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:40.763143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:40.776397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4307155-41e7-4657-a014-defe1726663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:40.777091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307155-41e7-4657-a014-defe1726663f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:40.777379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307155-41e7-4657-a014-defe1726663f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:40.777661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307155-41e7-4657-a014-defe1726663f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:40.778519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4307155-41e7-4657-a014-defe1726663f None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=c7881256809045a580c808ce222e1077) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:40.778840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4307155-41e7-4657-a014-defe1726663f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:40.780406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4307155-41e7-4657-a014-defe1726663f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:40.787870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 149/294] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:40 2022] GET /identity/v3/role_assignments?user.id=c7881256809045a580c808ce222e1077&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=58709cf0655a474eb4a33606ebb525df => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:41.844424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7f3b-46ff-4a6a-94d1-3ac03c700a12 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.845092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7f3b-46ff-4a6a-94d1-3ac03c700a12 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.845092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7f3b-46ff-4a6a-94d1-3ac03c700a12 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.846596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 146/295] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:41.852173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966835-fbd9-4951-92a3-3cb8b667da28 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.852678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966835-fbd9-4951-92a3-3cb8b667da28 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.852825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966835-fbd9-4951-92a3-3cb8b667da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.870224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2f966835-fbd9-4951-92a3-3cb8b667da28 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:41.871487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f966835-fbd9-4951-92a3-3cb8b667da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:41.876910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a207a1-1861-4c88-93cd-cb2e418cc6e5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.877218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a207a1-1861-4c88-93cd-cb2e418cc6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.879921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a207a1-1861-4c88-93cd-cb2e418cc6e5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.879921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 147/296] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:41.881386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643f4a-3f53-4d42-9115-e1742565ebac None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.881684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643f4a-3f53-4d42-9115-e1742565ebac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.881974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643f4a-3f53-4d42-9115-e1742565ebac None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.883506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 148/297] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:41.888709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a609-fd4b-40f1-b58f-12fdc420a230 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.888994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a609-fd4b-40f1-b58f-12fdc420a230 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.889274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a609-fd4b-40f1-b58f-12fdc420a230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.905883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d511a609-fd4b-40f1-b58f-12fdc420a230 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:41.907088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d511a609-fd4b-40f1-b58f-12fdc420a230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:41.946072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 150/298] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:41.947761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d1b65-3442-4155-a05f-05290cc98c5c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.948019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d1b65-3442-4155-a05f-05290cc98c5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.948316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d1b65-3442-4155-a05f-05290cc98c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.967550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b77d1b65-3442-4155-a05f-05290cc98c5c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:41.969032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b77d1b65-3442-4155-a05f-05290cc98c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:41.977102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 149/299] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:41.978333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec6d0-1fa9-4a28-943b-2d95277df930 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:41.978629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec6d0-1fa9-4a28-943b-2d95277df930 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:41.978911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec6d0-1fa9-4a28-943b-2d95277df930 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:41.980350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 150/300] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:41.980910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec6d0-1fa9-4a28-943b-2d95277df930 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.018377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.040307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 151/301] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:42.041896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019237-db95-4125-9dae-9e11939af1cb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.042193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019237-db95-4125-9dae-9e11939af1cb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.042477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019237-db95-4125-9dae-9e11939af1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.058172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-11019237-db95-4125-9dae-9e11939af1cb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:42.059682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-11019237-db95-4125-9dae-9e11939af1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:42.085439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec6d0-1fa9-4a28-943b-2d95277df930 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.101496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.118642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.119487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.119781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.120059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.127098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 152/302] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:42.127936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11019237-db95-4125-9dae-9e11939af1cb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.131135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:42.139028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:42.140072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:42.140946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 151/303] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:42.143972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.233631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11019237-db95-4125-9dae-9e11939af1cb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.240384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.246742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87435-d32b-4067-b77b-76bd0b226fc4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.268173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.269037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.269381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.269536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.281571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.281751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:42.282378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.282664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.282945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:42.289551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:42.290616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 153/304] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:42.291644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.296502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:42.305242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:42.306437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 152/305] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:42.308069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.330557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.395378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6bcc8-1e05-43a4-9aac-7a30e4abfd89 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.408385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb37e56-78ee-41ea-b513-9d9e902cc006 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.429266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.429950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.430233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.430510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.431295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:42.431607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.433033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.440891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 154/306] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.446329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.446434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.446434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.446434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.446434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.446761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.446761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.447591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 153/307] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.448339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.474081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.539283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.543060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd9b0d-3f92-41b5-9fed-ccad60826c4e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.549993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002685bf-67f0-4fd2-8fb1-29fda7b2b891 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.577691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.578360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.578641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.578912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.579688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:42.579997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.581400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.584894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.585569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.585855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.586132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.586872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:42.587246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.588612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.590069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 155/308] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.590837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.601476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 154/309] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.602259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.685047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.694782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b49a0f-ebaa-4e7f-a589-56b47d567237 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.700405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce166ec-039b-4346-a181-519f9bf701c9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.726919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.727831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.728114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.728383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] PATH_INFO: `/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.729557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] RBAC: Authorizing `identity:get_user(user_id=48a7445d5f874ed6815db1a8a0ce5fa1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.731025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.732696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 156/310] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/users/48a7445d5f874ed6815db1a8a0ce5fa1 => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.733437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.737807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.738485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.738769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.739368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.740062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.741522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.757462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 155/311] 10.222.0.65 () {60 vars in 1134 bytes} [Fri Jun 24 09:28:42 2022] POST /identity/v3/groups => generated 229 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:42.759016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.771037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:42.846547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2fcd8-658f-4225-8305-3424b2c073e5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.883988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ab256-149a-4529-a124-05f3ffec1536 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:42.884584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.885450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.885742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.886020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] PATH_INFO: `/v3/users/4c0c872427b540f3a3803e7a888535e8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.887369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] RBAC: Authorizing `identity:get_user(user_id=4c0c872427b540f3a3803e7a888535e8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.888880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.890621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 157/312] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/users/4c0c872427b540f3a3803e7a888535e8 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.891437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.926927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:42.928145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:42.928598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:42.929058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] PATH_INFO: `/v3/users/eece61aa21624dcfadc81d5a906fddec` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:42.945384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] RBAC: Authorizing `identity:get_user(user_id=eece61aa21624dcfadc81d5a906fddec)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:42.946887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:42.948859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 156/313] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/users/eece61aa21624dcfadc81d5a906fddec => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:42.954185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:42.958163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:43.001747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:43.009494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2b338-efb6-4c79-9337-463617f7160a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:43.054322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:43.055684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:43.056016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:43.056299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:43.060411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a61a-c098-474c-9b7f-c7a4bd97dc23 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:43.097241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:43.098042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:43.098387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:43.098674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] PATH_INFO: `/v3/system/users/4c0c872427b540f3a3803e7a888535e8/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:43.100281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c0c872427b540f3a3803e7a888535e8, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:43.101674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:43.119106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 157/314] 10.222.0.65 () {58 vars in 1334 bytes} [Fri Jun 24 09:28:42 2022] PUT /identity/v3/system/users/4c0c872427b540f3a3803e7a888535e8/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:43.119106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:43.142811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:43.144766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:43.203894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:43.250795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd2272-7351-4098-9a25-0ac95ffae755 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:43.281472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 158/315] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:43.282128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:43.289893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:43.293578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:43.294435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:43.294722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:43.294999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:43.296786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, user.id=eece61aa21624dcfadc81d5a906fddec, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:43.297224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:43.299229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:43.311163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 158/316] 10.222.0.65 () {56 vars in 1308 bytes} [Fri Jun 24 09:28:43 2022] GET /identity/v3/role_assignments?user.id=eece61aa21624dcfadc81d5a906fddec&scope.system=all&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:43.412385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6037f9f-7b00-49e9-8ba1-a7db327839c2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:43.449765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:43.450660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:43.450943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:43.451257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:43.452810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:43.454308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffbbe1-815d-4563-8727-85254966fd75 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:43.456455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 159/317] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:43 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:43.463016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:43.566826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9e005-1621-4884-b654-b9ddaa9c8a92 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:43.596418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:43.601589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:43.602267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:43.602551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:43.602829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:43.603709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, scope.project.id=ced62e933dfa4817922f06bf1c3b2183, user.id=48a7445d5f874ed6815db1a8a0ce5fa1) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:43.604026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:43.605480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432773b7-4c04-4e08-ab81-e287cbb0edf7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:43.612285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 159/318] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:28:43 2022] GET /identity/v3/role_assignments?user.id=48a7445d5f874ed6815db1a8a0ce5fa1&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:44.822663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41000eb-a7ce-41c2-9b58-1ce8152baaf7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:44.823238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41000eb-a7ce-41c2-9b58-1ce8152baaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:44.823238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41000eb-a7ce-41c2-9b58-1ce8152baaf7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:44.824792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 160/319] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:44.829872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:44.830288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:44.830509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:44.847970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:44.849186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:44.927113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 160/320] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:44.947207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41000eb-a7ce-41c2-9b58-1ce8152baaf7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.017040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.057680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41000eb-a7ce-41c2-9b58-1ce8152baaf7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.093881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.094750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.095041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.095345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.096545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:45.098120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5f220-1935-4ffa-9439-926c2053a74e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:45.100182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 161/321] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:45.105666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.210009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4895a-c75b-46a0-ae42-8fd98048292b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.229423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925359b3-de7e-487a-8444-feb0b4b159ae None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.229734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925359b3-de7e-487a-8444-feb0b4b159ae None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.229928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925359b3-de7e-487a-8444-feb0b4b159ae None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.231557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 162/322] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:45.236235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b41-62d4-4cbf-a82b-09db1b1b3d27 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.236491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b41-62d4-4cbf-a82b-09db1b1b3d27 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.236769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d5b41-62d4-4cbf-a82b-09db1b1b3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.238085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.248757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.249840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.250167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.250445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.251172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:45.252645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:45.255782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-846d5b41-62d4-4cbf-a82b-09db1b1b3d27 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:45.256986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-846d5b41-62d4-4cbf-a82b-09db1b1b3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:45.279209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 161/323] 10.222.0.65 () {60 vars in 1134 bytes} [Fri Jun 24 09:28:45 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:28:45.339323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 163/324] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:45.361050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.383026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88896576-e400-400c-932a-ed1d7ac58737 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.383387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88896576-e400-400c-932a-ed1d7ac58737 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.383615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88896576-e400-400c-932a-ed1d7ac58737 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.385027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 164/325] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:45.390662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.390945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.391267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.409312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:45.410671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:45.477159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.484857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9568c6-f722-4df7-85d5-cd131d3215fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.490604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 165/326] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:45.516034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.531109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.531859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.532259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.532406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.545281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:45.553688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:45.554600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:45.555621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 162/327] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:45.560887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.627125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.641975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c1437-9dee-434b-863a-9f0b4f006183 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.689891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.691430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.692088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.693568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.702251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99e5bd-f5fb-486b-bbb2-e27bad2500e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.710928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:45.719480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:45.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:45.721175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 166/328] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:45.727763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.765403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.766938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.767434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.767745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.768501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:45.768868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:45.770332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:45.782033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 163/329] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:45.789467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.820685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.884375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d8a0-6aaa-45c5-8bb6-f8291bfcc9a5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.910299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b5d7-a2e2-497e-8c88-4b0322e86dc6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:45.926074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:45.930393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.931189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.931696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.932037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.933251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:45.933645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:45.936182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:45.948647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 167/330] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:45.954242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:45.959970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:45.960914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:45.961195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:45.961472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] PATH_INFO: `/v3/users/4c0c872427b540f3a3803e7a888535e8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:45.962735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] RBAC: Authorizing `identity:get_user(user_id=4c0c872427b540f3a3803e7a888535e8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:45.964277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:45.966265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 164/331] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/users/4c0c872427b540f3a3803e7a888535e8 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:45.972003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:46.089857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff998c-32f9-4723-b2d4-33650037f303 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:46.105022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:46.111429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a75f9-29f4-4265-9963-f62ae7f3bd34 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:46.127581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:46.128483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:46.128762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:46.129034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] PATH_INFO: `/v3/users/eece61aa21624dcfadc81d5a906fddec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:46.131821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=eece61aa21624dcfadc81d5a906fddec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:46.133293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:46.134974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 168/332] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/users/eece61aa21624dcfadc81d5a906fddec => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:46.143987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:46.175911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:46.176894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:46.177191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:46.177473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:46.178330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] RBAC: Adding query filter params (role.id=58709cf0655a474eb4a33606ebb525df, user.id=4c0c872427b540f3a3803e7a888535e8, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:46.178648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:46.183128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187878d-d5ab-4b69-afdd-8b52ad98dc5a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:46.206794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 165/333] 10.222.0.65 () {56 vars in 1308 bytes} [Fri Jun 24 09:28:45 2022] GET /identity/v3/role_assignments?user.id=4c0c872427b540f3a3803e7a888535e8&scope.system=all&role.id=58709cf0655a474eb4a33606ebb525df => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:46.273546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d498-9368-4c79-84b8-9534497a6fc9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:46.314789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:46.315724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:46.316011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:46.316292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] PATH_INFO: `/v3/system/users/eece61aa21624dcfadc81d5a906fddec/roles/b16d427c48574ab1b79bcd0f1f571c27` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:46.324159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=eece61aa21624dcfadc81d5a906fddec, role_id=b16d427c48574ab1b79bcd0f1f571c27)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:46.325640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbcbc6-7fd3-4535-b97b-c92b637928ab None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:46.349009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 169/334] 10.222.0.65 () {58 vars in 1334 bytes} [Fri Jun 24 09:28:46 2022] PUT /identity/v3/system/users/eece61aa21624dcfadc81d5a906fddec/roles/b16d427c48574ab1b79bcd0f1f571c27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:47.998340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826302-626d-4fcf-a325-8a8919644421 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:47.998833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826302-626d-4fcf-a325-8a8919644421 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:47.998833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71826302-626d-4fcf-a325-8a8919644421 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.000752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 166/335] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:48.006181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.006576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.006751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.028936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:48.030143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:48.086862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:48.119686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 170/336] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:48.164613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71826302-626d-4fcf-a325-8a8919644421 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:48.203700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:48.277860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71826302-626d-4fcf-a325-8a8919644421 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:48.337672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:48.338504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.338800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.339091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.360469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:48.368389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:48.370473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:48.372475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 167/337] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:48.375918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:48.502337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d44b1-b045-4814-92aa-8f9192931b8e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:48.546712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:48.547357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.547644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.547925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.548682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:48.548995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:48.550419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:48.559433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 171/338] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:48.565133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:48.670750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:48.679489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4bef0-71b8-4cc0-abd1-9bbb21aed580 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:48.722606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:48.723531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.723822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.724101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.809922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:48.811364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb3cbd-9d83-4442-b668-a6967b5d2c3c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:48.919345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:48.934216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 168/339] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:48.939777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:48.942069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633699-a72f-431d-aead-ba3cb6bedb2e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.942835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633699-a72f-431d-aead-ba3cb6bedb2e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.942835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633699-a72f-431d-aead-ba3cb6bedb2e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.944261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 169/340] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:48.949740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:48.950032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:48.950316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:48.974731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:48.976145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:49.033183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:49.115352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 170/341] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:49.151999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.182000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928fe2-119c-47fe-a1e4-4472b0ac8ae4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.231641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.232533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.232816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.233092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.234331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:49.235816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a421-02fd-4f4f-bd0c-ade9720d9954 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:49.237717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 172/342] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:48 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:49.238744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a082075-f640-48cf-8b02-142d2e899dfe None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.239172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a082075-f640-48cf-8b02-142d2e899dfe None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.239739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a082075-f640-48cf-8b02-142d2e899dfe None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.242013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 173/343] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:49.248501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a082075-f640-48cf-8b02-142d2e899dfe None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.338655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8b051-7566-4142-bc19-2ba8fa9fc229 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.369272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:49.380740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a082075-f640-48cf-8b02-142d2e899dfe None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.381996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.382831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.384068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.384068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.389905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:49.405972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fc758399-c4f1-4c9d-9373-a7657a3ca162 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:49.417807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:49.418975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:49.419897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:49.421020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 171/344] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:49.422132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.422417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.422693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.424176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.425310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.425790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.426622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.433742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:49.435512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864eae9-382b-4572-8325-43140acd54b1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:49.437519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 174/345] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:49.438960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6f87f-a86f-4fe5-84fd-eba00548c0e1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.439490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6f87f-a86f-4fe5-84fd-eba00548c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.439988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6f87f-a86f-4fe5-84fd-eba00548c0e1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.441494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 175/346] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:49.443628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee8046-6b59-4749-9501-4c100d05c7d8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.444202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee8046-6b59-4749-9501-4c100d05c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.444681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee8046-6b59-4749-9501-4c100d05c7d8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.446179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 176/347] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:49.447307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee8046-6b59-4749-9501-4c100d05c7d8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.450815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:49.452211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:49.544448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 172/348] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:49.545660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.571339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee8046-6b59-4749-9501-4c100d05c7d8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.608706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.609671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.609671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.609871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.610607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:49.610916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:49.612425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001684f-aa1f-47f8-9274-fb71d463af90 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:49.621501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 177/349] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:49.622507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.622823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.623055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.650128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:49.651439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:49.662307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8ada5-965d-45a2-b8b2-54ca49e65dd9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.680330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:49.696637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:49.718795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.719521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.719940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.720391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.721732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=628cdb8129ea4f1191d369f8afd5294c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:49.722212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:49.724703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e24dc-168c-4b58-a9b0-7c0251f961ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:49.733846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 173/350] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=58709cf0655a474eb4a33606ebb525df => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:49.736252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.736971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.737872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.755641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 178/351] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:49.756554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.783665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:49.784897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:49.877029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 174/352] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:49.877692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:49.898487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb62456-dff7-4331-b210-912f9ce78ca4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:49.968676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:49.969809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:49.970092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:49.970368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:49.993636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:49.994890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc3cdd-e738-4cbb-89a4-4ca89c829474 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:50.001845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:50.002760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:50.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 179/353] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:50.005017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.015965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.038540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.071498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.073216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.073638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.074069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.144117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8844893-7466-4166-95f5-bab45bab86ab None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.158031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.159559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:50.192823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.194130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.194893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.195638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.209130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:50.217869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:50.218769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:50.219600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 180/354] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:50.220252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.281751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 175/355] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:50.282547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.295661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.326614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.351500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06751e8-cd41-4cb4-8f88-f036b576f2d6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.398094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.398851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.399198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.399477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.412925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:50.416876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38182e2a-7f52-413a-97a7-74ed3a442cec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ba055015-1398-4269-9ce8-98772719ed40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:50.421116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:50.421981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:50.423237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 181/356] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:50.424095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d42c2-5280-48a6-be16-3bcf0ae399ff None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.424168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d42c2-5280-48a6-be16-3bcf0ae399ff None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.424505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d42c2-5280-48a6-be16-3bcf0ae399ff None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.425706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 182/357] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:50.426830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d42c2-5280-48a6-be16-3bcf0ae399ff None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.466758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.467559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.467845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.468120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.469414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:50.469746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.472215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:50.484259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 176/358] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:50.487463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.556242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d42c2-5280-48a6-be16-3bcf0ae399ff None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.574389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.598908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.601386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.601672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.601945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.602707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:50.603016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.604466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:50.612991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b5f18-a4cb-4785-aa5e-702c67de676f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.613993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 183/359] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:50.614947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.618848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.657491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.658379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.658660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.658930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.660181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.661778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5391-de8d-4b38-ae94-eb87f827437f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:50.664934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 177/360] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:50.666739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.667033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.667364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.687506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:50.688854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:50.734139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c2be1-c325-4fde-a782-87d968e3df8e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.778938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 178/361] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:50.779971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.788718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.789711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.790719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.791293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.792703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:50.793209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.795590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:50.807990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 184/362] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:50.813237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:50.871511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.899393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:50.916010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38164c9f-a7a7-4d91-a97a-55efde9fa4a1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.944870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94badb55-5ff9-42df-8303-a07b12200751 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:50.963171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.964070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.964500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.964788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.992528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:50.993438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:50.993719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:50.994024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] PATH_INFO: `/v3/users/eece61aa21624dcfadc81d5a906fddec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:50.996568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] RBAC: Authorizing `identity:get_user(user_id=eece61aa21624dcfadc81d5a906fddec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:50.998700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.001400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 185/363] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/users/eece61aa21624dcfadc81d5a906fddec => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.002190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.047124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.048974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.129723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9871ab-ae30-48fc-a991-09f46b8b533d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.138070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.173776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.174372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.174372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.174658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.176687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.178156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.180045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 186/364] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.180671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.192308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.198336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 179/365] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:51.199246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.304253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949d316-207d-41c4-be10-52f26c165c5d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.333182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c7894-983c-4316-bb39-94c6bcac8a1f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.345361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.346304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.346681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.347050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.358384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:51.368306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:51.369209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 187/366] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:51.369974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.382490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.382713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.383112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.383112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.428119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.446559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.475943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.478103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.496624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bddb1-84a4-4894-973e-0619fb721389 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.538616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.539324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.539768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.540050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.541409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] RBAC: Adding query filter params (role.id=b16d427c48574ab1b79bcd0f1f571c27, user.id=eece61aa21624dcfadc81d5a906fddec, scope.system=all) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:51.541746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.543590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.561770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 188/367] 10.222.0.65 () {56 vars in 1308 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/role_assignments?user.id=eece61aa21624dcfadc81d5a906fddec&scope.system=all&role.id=b16d427c48574ab1b79bcd0f1f571c27 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.562367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.631961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 180/368] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:51.632774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.688786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bed68-80e4-4807-a279-fb9888ec7ead None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.717058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.725565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:51.737593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.738280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.738712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.739671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.740582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=628cdb8129ea4f1191d369f8afd5294c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:51.740910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.742417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.762443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 189/369] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=561eaf0348e94e0f924cd58981968818 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.763164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.790941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e1a96-9b1b-4cf0-9669-1bfe44729f9c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.845756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.846657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.846948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.847261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.848584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.850031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.853441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 181/370] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.854329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:51.931510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8f37-a82c-4926-9f8a-7db14b6e9f3d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:51.978293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:51.978913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:51.979235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:51.979519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:51.980292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:51.980605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:51.982042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:51.993137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 190/371] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:51.993930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.009904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203ab1b-ac6b-4f04-a17d-fb0558bf1df0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.050752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.052350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.052633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.053017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.054202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.055972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.056400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.057407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 182/372] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.058292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.063517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.119784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0941394-f637-4a56-a299-961e0ca7abf0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.159756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.162691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.162691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.162882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.164203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.166736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee85c93-40a5-4b2b-abcc-8b7ef385c2fc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.169934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 191/373] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:51 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.169934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca109-c05d-4a6e-9179-5ef4c7970bad None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.169934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca109-c05d-4a6e-9179-5ef4c7970bad None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.169934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca109-c05d-4a6e-9179-5ef4c7970bad None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.170750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 192/374] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:52.171314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca109-c05d-4a6e-9179-5ef4c7970bad None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.199836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a5188-f85f-4d76-a0a3-c7218e95b129 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.243284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.244083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.244466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.244871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.295662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca109-c05d-4a6e-9179-5ef4c7970bad None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.313809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.324834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.326320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3740f8-3b8b-4eb0-88cd-6409395f8404 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.330032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.339712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.340598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.340872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.341141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.342307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.343795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.345579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 193/375] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.346175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.439038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 183/376] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:52.440366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.440652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.440929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.454463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473aff7-48b9-4e27-8517-9c2c9e55cc1b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.459392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:52.460580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:52.489399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.490059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.490348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.490628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.491502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=ced62e933dfa4817922f06bf1c3b2183) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:52.491819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.493268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.507205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 194/377] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=58709cf0655a474eb4a33606ebb525df => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.507205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.542595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 184/378] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:52.543691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.570683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.583656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.644770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4ee04-cd9a-439f-b786-6ccfe9e4d1e5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.668438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0577dd-8f4e-4016-ae92-d9a1ac3b46ee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.688253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.688940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.689227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.689512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.690358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=ced62e933dfa4817922f06bf1c3b2183) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:52.691124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.693099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.702630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 195/379] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=561eaf0348e94e0f924cd58981968818 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.703899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.733892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.734794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.735108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.735386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] PATH_INFO: `/v3/projects/46a0c5caeb0244aaa5e89844285d900b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.736699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] RBAC: Authorizing `identity:get_project(project_id=46a0c5caeb0244aaa5e89844285d900b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.738204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.740160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 185/380] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/projects/46a0c5caeb0244aaa5e89844285d900b => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:52.748595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.871254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c58c7-b057-4cea-91c6-f0eb3c47231f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.882960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.917208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.917574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348190-ac64-430f-913e-f5c0ce3bd44b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:52.918053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.918341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.918620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.919318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:52.940599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:52.948516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:52.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:52.950626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 196/381] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:52.956132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:52.961709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:52.962661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:52.962957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:52.963259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] PATH_INFO: `/v3/groups/9c759a9548214087917e06ba95ebaae7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:52.975689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] RBAC: Authorizing `identity:get_group(group_id=9c759a9548214087917e06ba95ebaae7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:52.979023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:52.983103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 186/382] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/groups/9c759a9548214087917e06ba95ebaae7 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:52.991737 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:53.105219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849c55a-427b-4c32-933b-c89f181f2d97 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:53.145866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:53.146888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:53.147392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:53.147866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:53.148934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] RBAC: Adding query filter params (group.id=9c759a9548214087917e06ba95ebaae7, role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=46a0c5caeb0244aaa5e89844285d900b) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:53.149451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:53.151120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:53.159118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 187/383] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/role_assignments?group.id=9c759a9548214087917e06ba95ebaae7&scope.project.id=46a0c5caeb0244aaa5e89844285d900b&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:53.203043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebf2a7-7ef8-4733-a710-7ca8ad6d4d26 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:53.253855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:53.254602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:53.254889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:53.255202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:53.255974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:53.256284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:53.257714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:53.271582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 197/384] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:53.276832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:53.349333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:53.424039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580eca0-5357-4122-bda9-05d6a80072b1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:53.481155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:53.482344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:53.482807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:53.483309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:53.484683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:53.486304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:53.488153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 188/385] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:53 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:53.494461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:53.550448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:53.646190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de42e52-42f8-460a-8c80-50588fda1ab0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:53.700376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:53.701178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:53.701455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:53.701727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:53.708487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:53.709990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e343f-dfa0-4587-a7ff-6f006a9d7134 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:53.711856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 198/386] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:53 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:53.717491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:53.846614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:53.854988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bc740-62a0-41e1-96a0-c50358e48a31 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:53.902880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:53.903832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:53.904118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:53.904395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/groups/a3c10b8edcf84d009d54f6a0712df161/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:53.906446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, group_id=a3c10b8edcf84d009d54f6a0712df161, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:53.907944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248ff74-cf3b-4241-b22f-878c640bc2ad None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:53.935045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 189/387] 10.222.0.65 () {58 vars in 1442 bytes} [Fri Jun 24 09:28:53 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/groups/a3c10b8edcf84d009d54f6a0712df161/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:54.263402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25639-cd13-4a7f-88e5-7c11db88ef5c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:54.263941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25639-cd13-4a7f-88e5-7c11db88ef5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:54.263941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef25639-cd13-4a7f-88e5-7c11db88ef5c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:54.265604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 199/388] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:54.270584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:54.270877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:54.271178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:54.292078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:54.293273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:54.377512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 190/389] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:54.409280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef25639-cd13-4a7f-88e5-7c11db88ef5c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:54.561750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:54.563146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef25639-cd13-4a7f-88e5-7c11db88ef5c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:54.611482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:54.612314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:54.612598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:54.612872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:54.623843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:54.641120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:54.642032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:54.643069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 200/390] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:54.648961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:54.792796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:54.810395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfa7b3-bdac-4010-94ee-33caeb55d95e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:54.858728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:54.859337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:54.859629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:54.859903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:54.860654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:54.860971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:54.862379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:54.871543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 191/391] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:54.877749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.027752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174c627-c30c-4acf-91a1-08ea19e3bd1a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.074462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.075368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.075650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.075921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.077089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:55.078539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b48e-0ad4-4683-bd34-dcd8456842fb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:55.085193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 201/392] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:54 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:55.090466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.168370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac2dc0-0517-4058-94e4-a63692bfb956 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.169027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac2dc0-0517-4058-94e4-a63692bfb956 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.169532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac2dc0-0517-4058-94e4-a63692bfb956 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.171463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 202/393] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:55.180367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d7d0a-1dc7-4db9-81f2-bb8e723780af None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.180648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d7d0a-1dc7-4db9-81f2-bb8e723780af None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.180935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d7d0a-1dc7-4db9-81f2-bb8e723780af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.208992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-743d7d0a-1dc7-4db9-81f2-bb8e723780af None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:55.210467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-743d7d0a-1dc7-4db9-81f2-bb8e723780af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:55.240951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.253534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a25906-13b1-42ec-8730-b03392e4646d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.297058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.297847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 203/394] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:55.299832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813de3-120d-4a4a-8dfd-1de980304701 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.300296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813de3-120d-4a4a-8dfd-1de980304701 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.300698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813de3-120d-4a4a-8dfd-1de980304701 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.307157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.307157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.307157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.307597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 204/395] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:55.308827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:55.309381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.309675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.309953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.311049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:55.313301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 192/396] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:55.319676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.344025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:55.345753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:55.350426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.427256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 205/397] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:55.428612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.449458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f758e22-c8b9-4f34-b5f9-0ffbd3513856 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.495516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.496663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.497083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.497501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c/groups/a3c10b8edcf84d009d54f6a0712df161/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.500104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] RBAC: Authorizing `identity:create_grant(project_id=628cdb8129ea4f1191d369f8afd5294c, group_id=a3c10b8edcf84d009d54f6a0712df161, role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:55.501566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471eac-4417-4874-be75-50f6d1dbfa9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:55.516319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 193/398] 10.222.0.65 () {58 vars in 1442 bytes} [Fri Jun 24 09:28:55 2022] PUT /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c/groups/a3c10b8edcf84d009d54f6a0712df161/roles/561eaf0348e94e0f924cd58981968818 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:55.517625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47359-ef24-4e26-aaf9-8fcb2167bb28 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.517920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47359-ef24-4e26-aaf9-8fcb2167bb28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.518202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47359-ef24-4e26-aaf9-8fcb2167bb28 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.519645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 194/399] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:55.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd47359-ef24-4e26-aaf9-8fcb2167bb28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.537599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.555768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd77d6-ff7f-47d6-8422-d38debeb6698 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.597292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.598189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.598460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.598809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.612209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d98d49cd-9b4c-4f8f-a2fe-810f118d5cb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:55.621208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:55.622182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:55.622967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 206/400] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:55.624463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.624700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.624978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.644325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.646735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:55.648350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:55.671616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd47359-ef24-4e26-aaf9-8fcb2167bb28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.710041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.710931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.711250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.711527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.723068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:55.729442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 207/401] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:55.732225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:55.733075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:55.733863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 195/402] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:55.735424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.744866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.843450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.867442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce06b11-a195-43f7-8f87-5d49066513a0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.889117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a2f4-2e3e-4a91-9855-37e00207bce4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:55.907418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.908790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.909283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.909750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.910765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:55.911761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:55.913892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:55.925545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:55.926276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 208/403] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:55.926597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:55.926884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:55.927193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:55.927951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:55.928157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:55.928252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:55.929628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:55.936635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:55.942196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 196/404] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:55.943034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.053180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b80c36-c843-4f4f-80e2-eb0a411aa6c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.071036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe7f1-ea0e-49eb-bf71-a8cfe11454a0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.101374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.103157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.103645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.104323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.106377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.108457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.109902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.110778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.111151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.111316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 197/405] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.111539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.112604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.121148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.125034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:56.134915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:56.135845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:28:56.136624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:56.141607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 209/406] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:56.142624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.192284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.228768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec736-6a64-47f0-b43f-e2754949e331 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.250370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26bef1-a518-4246-9e58-a88c0fcc367c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.264255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.265281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.265682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.266268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.267346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.268895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.271011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 198/407] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.271902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.288263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.289354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.289771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.290179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.291605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.293291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.295446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 210/408] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.296438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.368201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.382583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927c2fc-fe61-4f19-81a2-df69773a7991 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.407584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d231e3-b313-4074-ae0b-20965948fa45 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.426267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.431182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.431533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.431884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.432623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:56.433158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.434432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.436913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.445424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.446628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.447116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.447583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.449176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.450870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.457826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 211/410] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.459029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.459965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 199/410] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.460844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.568238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823d7c6-b6fe-4717-8b94-b84d065d198b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.572501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bef27b-e2fb-47da-a4f7-65df6a6c68f4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.605889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.606780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.607060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.607364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/groups/a3c10b8edcf84d009d54f6a0712df161/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.609171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.609288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, group_id=a3c10b8edcf84d009d54f6a0712df161, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.610050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.610328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.610596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183/groups/a3c10b8edcf84d009d54f6a0712df161/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.610695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.612811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced62e933dfa4817922f06bf1c3b2183, group_id=a3c10b8edcf84d009d54f6a0712df161, role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.614945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9640a4-8c5a-4b71-9d88-8c4eab2f07f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.618586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.627632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 200/411] 10.222.0.65 () {58 vars in 1442 bytes} [Fri Jun 24 09:28:56 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/groups/a3c10b8edcf84d009d54f6a0712df161/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:56.628449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.643161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 212/412] 10.222.0.65 () {58 vars in 1442 bytes} [Fri Jun 24 09:28:56 2022] PUT /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183/groups/a3c10b8edcf84d009d54f6a0712df161/roles/561eaf0348e94e0f924cd58981968818 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:56.645791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe75b44-2916-4a14-9ce2-97c85a195e98 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.646416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe75b44-2916-4a14-9ce2-97c85a195e98 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.646946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe75b44-2916-4a14-9ce2-97c85a195e98 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.650489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 213/413] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:56.661951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.662596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.663107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.704378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:56.705672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:56.747105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:56.825213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e447e1-d5a4-4e4f-871b-9ef2c5e34fd3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:56.844287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 214/414] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:56.868699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.870440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:56.871541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:56.871998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:56.872458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] PATH_INFO: `/v3/groups/9c759a9548214087917e06ba95ebaae7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:56.873667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] RBAC: Authorizing `identity:get_group(group_id=9c759a9548214087917e06ba95ebaae7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:56.875463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:56.877171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 201/415] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/groups/9c759a9548214087917e06ba95ebaae7 => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:56.882885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:56.999349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef52a2-6d1f-407a-8b42-0fe02a1bc42a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.018113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b35a2-051f-4b94-833a-f19c34186809 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.050282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:57.059441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:57.065227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.066115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.066427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.066876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] PATH_INFO: `/v3/projects/46a0c5caeb0244aaa5e89844285d900b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.069456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] RBAC: Authorizing `identity:get_project(project_id=46a0c5caeb0244aaa5e89844285d900b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:57.071454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:57.073164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 202/416] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/projects/46a0c5caeb0244aaa5e89844285d900b => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:57.075614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.076486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.076765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.077036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.079838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:57.104030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:57.111772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:57.112639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:57.113491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:57.119300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 215/417] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:57.125456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:57.236007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d0bc-5966-4982-b33d-a0e79697be3a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.305030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffb935-16e8-45f6-bfc5-f13140ae0615 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.308265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.309411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.310127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.311463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] PATH_INFO: `/v3/projects/46a0c5caeb0244aaa5e89844285d900b/groups/9c759a9548214087917e06ba95ebaae7/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.318001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] RBAC: Authorizing `identity:create_grant(project_id=46a0c5caeb0244aaa5e89844285d900b, group_id=9c759a9548214087917e06ba95ebaae7, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:57.319837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:57.342309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 203/418] 10.222.0.65 () {58 vars in 1442 bytes} [Fri Jun 24 09:28:57 2022] PUT /identity/v3/projects/46a0c5caeb0244aaa5e89844285d900b/groups/9c759a9548214087917e06ba95ebaae7/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:28:57.360779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.361928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.362500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.362974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.364074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:57.364754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:57.366555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:57.377498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 216/419] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:57.384122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:57.451125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:57.507171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943618-5d53-44dc-a9ca-91a2f1a1b714 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.546226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.547314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.547874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.548348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.639389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:57.641205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65201d5-cb7a-4ef7-8bc2-0fbacf41de20 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:57.706119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:57.766108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 204/420] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:57.772035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:57.855002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:57.908936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20792fbc-ba5f-47d6-b582-6fb0e44bb270 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:57.953567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fef91-606d-4d84-933d-32742d29ac98 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.953904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fef91-606d-4d84-933d-32742d29ac98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.954132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fef91-606d-4d84-933d-32742d29ac98 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.955800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 205/421] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:28:57.960316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:57.961680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.962260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.962742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.964350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:57.965399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:57.965691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:57.965968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:57.966721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:57.969078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 217/422] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:57 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:57.974245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:58.016912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:58.018902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:58.092682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1081e-dcd0-4081-ae66-69704981cc47 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:58.131814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:58.169916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 206/423] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:58.180266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:58.181228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.181548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.181884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.188479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:58.189365 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:58.191816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 218/424] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:57 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:58.197524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:58.228802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:58.293266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:58.309753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd079da-e781-4891-acc1-319da526a5c0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:58.350084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:58.350762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.351200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.354426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.355036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=628cdb8129ea4f1191d369f8afd5294c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:58.355394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:58.356870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:58.366217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 207/425] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:58 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=58709cf0655a474eb4a33606ebb525df => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:58.435009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bc000-8fe2-4157-9b0b-30409fe0d267 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:58.483799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:58.484648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.484935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.485214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.503787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:58.512352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:58.513268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:58.514824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 219/426] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:58.519519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:58.666325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e86dd-9800-46ca-8aed-3f12cfe34065 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:58.714389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:58.714997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.715314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.715595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.716358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:58.716671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:58.718152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a29658-8e57-4b24-81e9-d07dfb72c535 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:58.731648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 208/427] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:58.736769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:58.795827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:58.858566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dc0b0-957c-4854-9e66-1b252e937682 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:58.899725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:58.900400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.900683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.900961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.974977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:58.976521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:58.995407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b800f56-5bfa-4f53-93be-96ae1982d51d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:58.996032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b800f56-5bfa-4f53-93be-96ae1982d51d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:58.996505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b800f56-5bfa-4f53-93be-96ae1982d51d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:58.999849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 209/428] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:59.004600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.005163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.005638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.025699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.029034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:59.030453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:59.034243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.120113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 220/429] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:28:59.125174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 210/430] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:28:59.125869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.148169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.259582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934bd44-e0bb-47c4-8d2f-b670a59a581d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.268312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23964a7-9027-4e85-82f2-027bc39cf3c5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.306065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.306924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.307044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.307250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.307529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] PATH_INFO: `/v3/projects/628cdb8129ea4f1191d369f8afd5294c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.308400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.308738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] RBAC: Authorizing `identity:get_project(project_id=628cdb8129ea4f1191d369f8afd5294c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.308978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.309398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.310231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc4a3f-7864-401a-9d54-4cb861fd599e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.313463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 221/431] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/projects/628cdb8129ea4f1191d369f8afd5294c => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:28:59.314446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757dca7-e098-45e0-a522-5b10678561d5 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.314735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757dca7-e098-45e0-a522-5b10678561d5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.315014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757dca7-e098-45e0-a522-5b10678561d5 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.317017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 222/432] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:28:59.318470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757dca7-e098-45e0-a522-5b10678561d5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.320647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.325266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:59.334413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-da621146-4d68-4f11-8e5e-cb566aff6ba9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:59.335776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:59.336723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 09:28:59.337540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:28:59.341075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 211/433] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:28:59.343487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820e19-b744-4401-8089-77932d298fbf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.343787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820e19-b744-4401-8089-77932d298fbf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.344072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820e19-b744-4401-8089-77932d298fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.361003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df820e19-b744-4401-8089-77932d298fbf None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:28:59.362194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df820e19-b744-4401-8089-77932d298fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:28:59.435160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 212/434] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:28:59.435921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df820e19-b744-4401-8089-77932d298fbf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.446782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757dca7-e098-45e0-a522-5b10678561d5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.484471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.485355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.485642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.485923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.487125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.488701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.490364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 223/435] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:59.491144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.556064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df820e19-b744-4401-8089-77932d298fbf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.573957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.587091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.593830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.594519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.594801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.595103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.595876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:59.596181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.597633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.598358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fad3cf-27ea-4d0b-b240-4ff6c06bda9b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.606232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 213/436] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:59.607058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.641449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.642345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.642558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.643002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.653517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:28:59.661982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:28:59.663005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 224/437] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:28:59.664112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.720251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497f78-dd44-4476-955f-18b040e0a9a6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.760342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.761293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.761905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.762191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.763344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=628cdb8129ea4f1191d369f8afd5294c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:59.763663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.765555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.771224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5d65-ab2f-4bbf-b637-d84840186f88 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.779332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 214/438] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=628cdb8129ea4f1191d369f8afd5294c&role.id=561eaf0348e94e0f924cd58981968818 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:59.779332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:28:59.829566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.830338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.830729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.831134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.836993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.850985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:28:59.891683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b50f2-f1f9-4e3f-9588-ded07815fc87 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:28:59.931831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:28:59.932673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:28:59.933028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:28:59.933423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:28:59.934257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:28:59.934632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.936184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.938493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:28:59.940052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82ed56-97d2-443a-bb40-0fbab9138d5e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:28:59.946400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 215/439] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:28:59.952079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.051569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 225/440] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:00.053073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6714a-8ea3-469a-b6c6-d4ffeba1ad37 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.053355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6714a-8ea3-469a-b6c6-d4ffeba1ad37 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.053631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6714a-8ea3-469a-b6c6-d4ffeba1ad37 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.059006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 226/441] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:00.059828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6714a-8ea3-469a-b6c6-d4ffeba1ad37 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.065814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acecf0-db70-44a1-861f-688a254dfc57 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.085382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.109026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.109695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.109982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.110262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.113454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.186010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6714a-8ea3-469a-b6c6-d4ffeba1ad37 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.186264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.187803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.219275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.220162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.220453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.220724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.221927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.223466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc3a7-34b5-4c1e-8c5c-c2f24d0be765 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.225172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 227/442] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.226304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.226588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.226866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.242462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:00.243688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:00.292977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 216/443] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:29:00.293945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.310037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 228/444] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:00.311534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.318290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.362489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.399487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902b964-c2e2-4090-99df-91b956e5efdb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.422849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cea6c-3a52-472b-884f-6a79e7b9ec17 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.438411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.439314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.439588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.439920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.441089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.443191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.444847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 217/445] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.445615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.462627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.463554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.463836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.464114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] PATH_INFO: `/v3/projects/ced62e933dfa4817922f06bf1c3b2183` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.465297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] RBAC: Authorizing `identity:get_project(project_id=ced62e933dfa4817922f06bf1c3b2183)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.466759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.468531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 229/446] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/projects/ced62e933dfa4817922f06bf1c3b2183 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.469275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.537312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.552689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53abf1-65e5-4c0e-9047-00276201caa9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.570583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211e6a-090a-4e41-83f5-e1627e34723a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.577813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.587667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.588476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.588765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.589048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.599228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:00.605784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.606441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.606720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:00.606810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:00.607660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:00.608438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:00.609036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.609036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=58709cf0655a474eb4a33606ebb525df, scope.project.id=ced62e933dfa4817922f06bf1c3b2183) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:00.609152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 218/447] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:00.609230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.609297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.609945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.618119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 230/448] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=58709cf0655a474eb4a33606ebb525df => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.618831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.730145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46020b3-732a-4c39-b911-9c21070fcfac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.733350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a76b7-b869-4277-8240-e83121a39512 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.774523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.775465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.775750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.776030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] PATH_INFO: `/v3/groups/a3c10b8edcf84d009d54f6a0712df161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.776333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.777035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.777261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] RBAC: Authorizing `identity:get_group(group_id=a3c10b8edcf84d009d54f6a0712df161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.777360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.777559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.778333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:00.778700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.778805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.780150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.781097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 219/449] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/groups/a3c10b8edcf84d009d54f6a0712df161 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.786760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.789916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 231/450] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:00.795791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:00.806359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.836638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:00.903063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac275a2-e01c-4623-bd01-4d1597fb2271 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.924569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1558fc-f99e-4ac0-a1bb-179fe804aad0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:00.943986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.945009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.945501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.945973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:00.947027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] RBAC: Adding query filter params (group.id=a3c10b8edcf84d009d54f6a0712df161, role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=ced62e933dfa4817922f06bf1c3b2183) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:00.947566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:00.949238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:00.956532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 220/451] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/role_assignments?group.id=a3c10b8edcf84d009d54f6a0712df161&scope.project.id=ced62e933dfa4817922f06bf1c3b2183&role.id=561eaf0348e94e0f924cd58981968818 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:00.960839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:00.961807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:00.962093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:00.962373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:01.036353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:01.037850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:01.069755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:01.145622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 232/452] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:01.151595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:01.230383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:01.252299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60cf5e-030b-4bc5-9e20-418a580a9aa3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:01.289760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:01.290630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:01.290917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:01.291235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] PATH_INFO: `/v3/projects/46a0c5caeb0244aaa5e89844285d900b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:01.292402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] RBAC: Authorizing `identity:get_project(project_id=46a0c5caeb0244aaa5e89844285d900b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:01.293871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:01.295598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 221/453] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:29:01 2022] GET /identity/v3/projects/46a0c5caeb0244aaa5e89844285d900b => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:01.301291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:01.409656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7912dd-0c94-43c0-9ea1-c2f8d1ea149c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:01.437307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:01.444597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:01.445892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:01.446233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:01.446606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] PATH_INFO: `/v3/groups/9c759a9548214087917e06ba95ebaae7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:01.448757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] RBAC: Authorizing `identity:get_group(group_id=9c759a9548214087917e06ba95ebaae7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:01.451069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927921-a179-449c-8c6e-c9366d703d7d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:01.453299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 233/454] 10.222.0.65 () {56 vars in 1180 bytes} [Fri Jun 24 09:29:01 2022] GET /identity/v3/groups/9c759a9548214087917e06ba95ebaae7 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:01.460858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:01.566363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc41d5-de39-45b8-89a4-f0bcbd45e856 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:01.602890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:01.603577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:01.603859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:01.604146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:01.604972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] RBAC: Adding query filter params (group.id=9c759a9548214087917e06ba95ebaae7, role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=46a0c5caeb0244aaa5e89844285d900b) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:01.605284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:01.606721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6d535-6b0d-4770-bdf9-8374a8c396c4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:01.614274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 222/455] 10.222.0.65 () {56 vars in 1376 bytes} [Fri Jun 24 09:29:01 2022] GET /identity/v3/role_assignments?group.id=9c759a9548214087917e06ba95ebaae7&scope.project.id=46a0c5caeb0244aaa5e89844285d900b&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:04.016446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8719bd-1140-4773-8a21-91102f9222bf None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.017451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8719bd-1140-4773-8a21-91102f9222bf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.017930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8719bd-1140-4773-8a21-91102f9222bf None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.019835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 234/456] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:04.025481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaa69-b980-4714-a482-fc3823538baa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.026291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaa69-b980-4714-a482-fc3823538baa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.026762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaa69-b980-4714-a482-fc3823538baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.052117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-acfbaa69-b980-4714-a482-fc3823538baa None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:04.053652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-acfbaa69-b980-4714-a482-fc3823538baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:04.056934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:04.140989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 223/457] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:04.165873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8719bd-1140-4773-8a21-91102f9222bf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.220524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911063-d71b-44a9-94ea-102b6b6c1a4b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.221158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911063-d71b-44a9-94ea-102b6b6c1a4b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.221660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911063-d71b-44a9-94ea-102b6b6c1a4b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.223323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 224/458] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:04.228437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.228954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.229421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.252186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:04.254509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:04.322681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8719bd-1140-4773-8a21-91102f9222bf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:04.374895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 225/459] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:04.385313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:04.386455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.386930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.387422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.398259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:04.400004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:04.411308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:04.432937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:04.433834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:04.434666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:04.434666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:04.434666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:04.434666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:04.445872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 235/460] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:04.452746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.568883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:04.570266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:04.601214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b1858-3d6e-49f4-a564-4d26c293cba1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:04.643759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:04.644911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.645341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.645755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.656741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:04.657883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75980-1453-4e30-b295-82a8963a1072 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:04.658607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ae1d1076-7391-45fa-ab99-8296c18c8ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:04.677921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:04.679026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:04.680119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:04.680119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:04.680119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:04.680119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:04.681998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 226/461] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:04.683547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05d81d-3f41-46dd-ae6e-98b3632083aa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.683972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05d81d-3f41-46dd-ae6e-98b3632083aa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.684373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05d81d-3f41-46dd-ae6e-98b3632083aa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.689825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 227/462] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:04.690834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05d81d-3f41-46dd-ae6e-98b3632083aa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.700349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:04.701263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.701696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.702119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.703093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:04.703539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:04.705088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de4250-7884-442f-8219-04585dc37b31 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:04.717225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 236/463] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:04.718682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36b65-2673-4342-8fe9-be122ac9c541 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.719169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36b65-2673-4342-8fe9-be122ac9c541 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.719600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36b65-2673-4342-8fe9-be122ac9c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.736661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-eae36b65-2673-4342-8fe9-be122ac9c541 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:04.737863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eae36b65-2673-4342-8fe9-be122ac9c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:04.807273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05d81d-3f41-46dd-ae6e-98b3632083aa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:04.817247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 237/464] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:04.818895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5c05-3b41-4541-9219-6a3e619fab68 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.819228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5c05-3b41-4541-9219-6a3e619fab68 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.819504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5c05-3b41-4541-9219-6a3e619fab68 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.820797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 238/465] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:04.821696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd5c05-3b41-4541-9219-6a3e619fab68 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.842504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:04.856857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:04.858172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.858696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.859156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.860492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:04.861733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:04.863893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c8981-fdbd-4c5d-8d9a-1ba004c07265 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:04.874056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:04.885028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 228/466] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:04.885028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.885028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.885028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.901357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:04.902552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:04.938226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd5c05-3b41-4541-9219-6a3e619fab68 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:04.974582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 229/467] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:04.974905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:04.975914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:04.976154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:04.976516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:04.977047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:04.977885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:04.979557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.011206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 239/468] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:04 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:05.011206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.105784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06273110-3998-4f5d-bc29-ea40aa6eaa9d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.122063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:05.146858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:05.155267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242e953-1e42-41d8-b9bc-eb0f2ad94e4d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.157104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.159298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.159766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.160212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.172085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.173844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:05.193314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:05.194205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:05.195068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:05.195068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:05.195068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:05.195068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:05.199392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 230/469] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:05.200247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.202102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.202317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.202585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.202862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.203878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.205059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.229048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 240/470] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:05 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:05.229937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.339623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653f37-32ff-4892-a80c-c94b780c4cae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.373301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3e8d5-c28e-440e-a41d-2616062d92b4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.391688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.392704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.393053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.393397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.408099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.409993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.426904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.427641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.427926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.428200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.428938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:05.429247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.430645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:05.433935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:05.434820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:05.435712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:05.435712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:05.435712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:05.435712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:05.436848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 231/471] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:05.443899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.448318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 241/472] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:05.453717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.467921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:05.476573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:05.566652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bbfb6-7eb3-4b47-8ff5-fc53c8e40f62 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.584077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58fbce-9722-42bb-ba4f-fe41f33f1c7c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.614372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.615574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.615922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.616290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.617096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.619692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.630844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.631496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.631784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.632165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.632911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:05.633318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.634746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80195e54-cafc-460b-82c7-c214acaff989 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.649527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 242/473] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:05 2022] POST /identity/v3/users => generated 249 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:29:05.649936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 232/474] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:05.655752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:05.747582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:05.794718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1745-921d-4014-a7f8-d29cc44d0d4b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:05.841583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:05.842288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.842572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.842850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.843737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:05.845333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6d48c-6060-4107-818b-3e645abd3257 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:05.872094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 243/475] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:05 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:29:05.913601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9ca5-499c-4d0f-af2f-186eb739536c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.913982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9ca5-499c-4d0f-af2f-186eb739536c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.914324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c9ca5-499c-4d0f-af2f-186eb739536c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.916269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 233/476] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:05.922439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:05.922752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:05.923011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:05.946335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:05.948431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:06.054099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 244/477] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:06.106498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c9ca5-499c-4d0f-af2f-186eb739536c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:06.210269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:06.236922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c9ca5-499c-4d0f-af2f-186eb739536c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:06.284455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:06.285684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:06.285772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:06.286203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:06.286898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:06.288362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:06.298973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:29:06.298973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 09:29:06.298973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [parameters: {'id': '6cb3576eda884f9e95d752f65aea0687', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 09:29:06.298973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:06.322845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:06.325091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:29:06.326254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:29:06.327378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:06.328775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:06.330590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'id': '6cb3576eda884f9e95d752f65aea0687', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:06.336194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 09:29:06.337204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:06.338210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 234/478] 10.222.0.65 () {60 vars in 1131 bytes} [Fri Jun 24 09:29:06 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:06.344233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:06.407336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:06.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce896542-f884-44f3-8b2f-409ac6802789 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:06.520336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:06.521664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:06.522184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:06.522683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:06.542835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2a452f2f-7f64-4cd2-833a-75676cd06f44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:06.552057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:06.553133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 09:29:06.553910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:06.556740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 245/479] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:06.562513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:06.737761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3321-03c6-4014-8265-9b9aa988b1af None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:06.800533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:06.800533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:06.800533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:06.800533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:06.800956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:06.800956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:06.800956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41322a-2a87-45a5-ae50-f57a3522ae63 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:06.808177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 235/480] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:07.588122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f759c-72b6-43ff-8c4a-dcea1c346023 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:07.589127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f759c-72b6-43ff-8c4a-dcea1c346023 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:07.589562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f759c-72b6-43ff-8c4a-dcea1c346023 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:07.591430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 246/481] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:07.604619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741bec1-5842-48ae-a875-676f90bc38cf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:07.605466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741bec1-5842-48ae-a875-676f90bc38cf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:07.605891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741bec1-5842-48ae-a875-676f90bc38cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:07.611644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:07.628277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3741bec1-5842-48ae-a875-676f90bc38cf None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:07.630473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3741bec1-5842-48ae-a875-676f90bc38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:07.717137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 236/482] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:07.753024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f759c-72b6-43ff-8c4a-dcea1c346023 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:07.873182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f759c-72b6-43ff-8c4a-dcea1c346023 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:07.897274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:07.922633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:07.923491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:07.923780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:07.924062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:07.935000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:07.945428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba9b-89c3-428e-b6c2-2af3fc2ea053 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:07.945723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba9b-89c3-428e-b6c2-2af3fc2ea053 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:07.945995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba9b-89c3-428e-b6c2-2af3fc2ea053 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2bbf706b-d0e2-4dfa-b97b-a456cefa5aef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:07.947351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:07.948460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:07.949808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:07.950403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 237/483] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:07.950478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 247/484] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:07.954447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aba9b-89c3-428e-b6c2-2af3fc2ea053 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:07.958453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:07.958798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:07.959099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:07.990344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:07.991997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:08.085190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aba9b-89c3-428e-b6c2-2af3fc2ea053 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.105446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 248/485] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:08.128148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.139520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.140491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.141010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.141494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.142438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:08.142933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.144658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.155853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 238/486] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:08.157141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.168553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.298328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc829e-5e20-4e9a-8fdb-491e491eb2cd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.306735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ead47-0056-49ce-ab2d-0e3dc038c09e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.307005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.338938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.340211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.340771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.341191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.341466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.341999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.342291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.342554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.353337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:08.354735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.356252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3227acc2-d86a-4f51-a712-28c26fe86e34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:08.362170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:08.363189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:08.363989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:08.366463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 249/487] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:08.372535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab03953-d23c-4f55-82a2-9226bdd5e62d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.373064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab03953-d23c-4f55-82a2-9226bdd5e62d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.373531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab03953-d23c-4f55-82a2-9226bdd5e62d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:08.374578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:08.375404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:08.376934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:08.376934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:08.376934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:08.376934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:08.376934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 239/488] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:08.377309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.378830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 250/489] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:08.381911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab03953-d23c-4f55-82a2-9226bdd5e62d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.430381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.489888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ace27-5a75-44e8-afb2-737a24fd268c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.512853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab03953-d23c-4f55-82a2-9226bdd5e62d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.530367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.531564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.532046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.532513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.533540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:08.534169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.536401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb78de-ad9f-4e76-bf4c-eb52bccc1ccd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.551291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 240/490] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:08.552622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.552917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.553197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.556344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.557026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.557307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.557582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.558320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:08.558624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.560063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.576001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:08.577732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:08.577837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 251/491] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:08.578644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.606961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.651719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 241/492] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:08.653405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.682261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a85a7-8334-490e-8ad5-9be6ddadad61 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.697621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.719423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.720567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.721143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.721638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.732458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.734191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:08.754705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:08.755732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:08.756571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:08.756571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:08.756571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:08.756571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:08.758570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 252/493] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:08.759588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.768434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ec854-3b29-4878-a9fc-b8a7aa778fc4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.803489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.804319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.804655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.804933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.812443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.813957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:08.830261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:08.831220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:08.832167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:08.836421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 242/494] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:08.837288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.863886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.874385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4baa81-b86f-47d7-9799-1dc5d34d29d3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.911086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.912033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.912376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.912729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.923619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:08.931433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:08.932345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:08.933164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 253/495] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:08.933805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:08.945048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:08.948255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30386c-a981-4619-a1f5-c2bf5e71de20 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:08.985918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:08.986579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:08.986882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:08.987181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:08.987950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:08.988261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:08.989751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8220a-8bdf-4df8-81c1-83fc8061d991 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.002017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 243/496] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.003144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b27-e968-4ff3-b19a-28aed418d80c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.003490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b27-e968-4ff3-b19a-28aed418d80c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.003940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b27-e968-4ff3-b19a-28aed418d80c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.009118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 244/497] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:09.009118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85b27-e968-4ff3-b19a-28aed418d80c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.045086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04123320-e97c-4100-b4fe-ba6d9dfba0ac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.083455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.084198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.084480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.084750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.085633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:09.085941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.088044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.112244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.112695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85b27-e968-4ff3-b19a-28aed418d80c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.134206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 254/498] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.135066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.160103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.160787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.161076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.161421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.162188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:09.162404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.163832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45985-7eb0-47f9-93d3-2da8a34045b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.172480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 245/499] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.178564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.178855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.179156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.196040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:09.197225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:09.211613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.245123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb9e50-9269-4ec4-95dc-c639d5ef99c7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.269183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 246/500] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:09.270391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.286645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.287764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.288216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.288703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.296882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.298463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:09.314426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:09.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:09.316765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:09.317971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 255/501] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:09.322951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.389489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e00c94-f266-4185-9063-f2c41662f4b4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.394821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.426391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.426961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.427279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.427558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.444143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0abd-29d7-4b98-babf-73a996509325 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.446358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.489864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.490709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.490968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.491280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.498294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.500181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.503741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.505163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:09.522850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:09.524011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:09.525039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:09.525039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:09.525039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:09.525039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:09.525438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 256/502] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:09.526259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.599142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 247/503] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:09.599994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.627604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c9d0-ff84-45f3-98e3-f32ac508e1fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.637501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.650157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.662661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.663495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.663775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.664044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.673657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-62823e97-af4c-42c4-8c10-dad0bf43e233 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:09.682026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:09.682885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:09.683907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 257/504] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:09.684585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339ac9-3da6-4657-b25f-d4d8b18b62d8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.684585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339ac9-3da6-4657-b25f-d4d8b18b62d8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.684843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53339ac9-3da6-4657-b25f-d4d8b18b62d8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.686114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 258/505] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:09.686794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53339ac9-3da6-4657-b25f-d4d8b18b62d8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.705531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd142f-4bd1-4bd8-b6f8-2ef9a0683582 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.740078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.740755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.741036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.741314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.742158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:09.742469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.743954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.781454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 248/506] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.782136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.791904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53339ac9-3da6-4657-b25f-d4d8b18b62d8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.824945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.825621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.825907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.826183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.826910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:09.827249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.828668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.840141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 259/507] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.840951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:09.873144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.889546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a1e9f-25a1-4cc7-ae00-82dabde6ea18 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.898903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:09.926449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.927283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.927566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.927839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.938411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.939910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.945901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c6532-7665-4214-8898-052f2be5f53a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6189ad39-8c9c-4095-ad6d-ed129f4f7262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:09.957470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:09.958328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:09.959460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:09.959460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:09.959460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:09.959460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:09.959460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 249/508] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:09.959910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.960200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.960473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.977120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:09.978290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:09.985245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:09.985929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:09.986210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:09.986484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:09.987242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:09.987569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:09.989007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:09.997167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 260/509] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:09.998355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.046560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 250/510] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:10.047434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.097635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.108579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429504-c8c9-4627-a425-14605e77c47f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.142915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.143498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.143873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.144629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.144864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.153035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88de58f-4382-4717-9691-0c2a5b03338c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.187452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.188258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.188597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.188827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.196177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.197647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.209671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.211155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:10.212104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:10.212963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:10.213833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 251/511] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:10.214364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.307664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.318692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 261/512] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:10.319547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.322730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f76e-d54e-469a-a526-b1be146c024d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.356758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.357431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.357717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.357994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.358717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:10.359027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.360464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.372194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 252/513] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:10.373096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.388426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.422100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985587f-f8b7-4b35-9fe7-61aff836af1d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.456245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.457418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.457750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.457999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.468093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.469559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.479166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fc9d3-fbf0-4774-a535-289e96e4ceff None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:10.487371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:10.488674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 262/514] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:10.494222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.515632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.516451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.516739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.517019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.527984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.529461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.533814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:10.546645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:10.547688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:10.548759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:10.548759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:10.548759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.548759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:10.552923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 253/515] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:10.553752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.590703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5222a4-a1b8-4455-abd6-fe01c285ea1f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.616553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.624976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.625730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.626013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.626291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.627172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:10.627483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.629010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.646838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 263/516] 10.222.0.65 () {56 vars in 1139 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:10.647953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.660770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d102d-7f75-468e-a349-a6aa2c9ab3ac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.694671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.695501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.695785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.696055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.706572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.708165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:10.725357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:10.726247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:10.727095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:10.727095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:10.727095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:10.727095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:10.727095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 254/517] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:10.727733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.749965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.753215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a082-7356-484a-8d61-70b8a9fda0f9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.789240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.790047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.790329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.790615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.797044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.798527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:10.813060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:10.813887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:10.814698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 264/518] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:10.815185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.832806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c519c7-fd93-4ab2-97da-9136f91081fa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.846834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:10.867718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.868383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.868662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.868923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.869643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:10.869950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.871365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.883694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 255/519] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:10.884478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.924299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ee956-6b3d-4bb5-a51e-cc974db2cd11 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.957658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:10.958262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:10.958546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:10.958819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:10.959581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:10.959891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:10.961374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:10.973026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 265/520] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:10.973815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:10.990748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb935ea-cda3-49ea-a704-e284c29113d4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:10.994718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.025184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.025834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.026118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.026399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.071479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff08983-dd08-4912-aca2-f2be580d1ae9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.071748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.098867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.100479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.104914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.105572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.105851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.106123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.106832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:11.107161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.108529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.119873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 266/521] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:11.120765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.201868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 256/522] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:11.202736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.206928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.222473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffbef-cf22-46e4-a2c3-93ba0b144c21 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.257291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.257966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.258253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.258533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.259363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:11.259680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.261183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.274433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.278950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 267/523] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:11.279771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.305444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd1af0-876a-48ef-a122-42a45c58166f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.338020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.338825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.339138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.339431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.346726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.348230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:11.363108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:11.364137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:11.365187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 257/524] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:11.365749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.383789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45097af-5fde-4b62-a29a-93ada7d046bc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.418906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.419582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.419886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.420160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.420856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.422328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.432569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.444642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 268/525] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:11 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:11.445651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.468230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02dcfa-f792-431e-92e0-187332c7c8a9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.508557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.509291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.509577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.509855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.512082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.518303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.520236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:11.536671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:11.537822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:11.538677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 258/526] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:11.539144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.583816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234113af-c8f5-4d7a-924a-561cd93d1f53 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.625621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.626540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.626890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.628017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.639730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.641327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:11.660705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:11.661820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:11.663017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:11.663017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:11.663017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:11.663017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:11.663434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b524f-d540-4f20-b710-6495b564ba39 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.663507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 269/527] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:11.663507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.702014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.702685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.702971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.703290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.704131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:11.704449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.705919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.711744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 259/528] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:11.712529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.721675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.785875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a487d86-f634-4156-875e-130a31739ed1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.791390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:11.843905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.844909 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.845344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.845529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73b2cd-4952-494d-89b1-d096dff1b1af None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:11.845894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.846889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:11.847721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.850507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.875697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 270/529] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:11.876925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:11.886564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:11.887283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:11.887640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:11.887991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:11.888874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:11.889641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:11.891233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:11.911926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 260/530] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:11.912579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.005062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1351d1-f49e-45df-a9e9-68aa7ff994d3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.015298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.036938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368b69f-0dfd-4f17-aece-c3c473e9fba4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.051508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.052440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.052896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.053342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.054246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:12.054757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.056377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.057095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.075186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 271/531] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:12.076077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.080284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.081846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.082133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.082497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.170647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.172597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.184779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec5442-b932-4349-91f8-88c2eade66ce None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.221278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.222675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.223209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.223793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.224845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=01aa5644b3834b0d887bbfc854ebc8cd) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:12.225368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.227260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.233892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 272/532] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/role_assignments?user.id=01aa5644b3834b0d887bbfc854ebc8cd&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:12.235324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.303716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 261/533] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:12.309382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.324833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.329894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.420902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4081eb-0b22-494b-9832-1efa0c184461 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.465061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.470831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.471292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.473161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.485063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.486654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:12.508552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:12.509432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:12.510276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:12.510276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:12.510276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:12.510276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:12.511059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b575-6f52-4b99-a41c-281095feaab9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.514520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 262/534] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:12.515886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.551797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.552964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.553455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.554083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.562041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.563850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:12.580472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:12.581335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:12.582905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:12.585382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 273/535] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:12.591236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.633170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.700577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e79f-c98c-4c83-86c0-bb26ddba1cb5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.768763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:12.770315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.772355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.773510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.773982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.774739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:12.775044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.776463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.791500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac22c6c-0536-41a0-95ed-df71010bb8a2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:12.806730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 263/536] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:12.809500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.853478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:12.854551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:12.855166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:12.856475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:12.858142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:12.858831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:12.861750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:12.889397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 274/537] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:12.899111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:12.989468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf236a-e39e-47f6-b170-1ac06124d6f1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:13.051431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:13.055766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:13.056516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.056856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.057086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.071318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:13.072828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:13.100746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1821ba-1693-49e5-9bdd-036defa339ab None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:13.103763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:13.104604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:13.105444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:13.109554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 264/538] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:13.115021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:13.163377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:13.165365 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.165652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.165930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.166778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=2381343893e74c0a919ff4d0d59b655c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:13.167111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:13.168579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:13.188155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 275/539] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:12 2022] GET /identity/v3/role_assignments?user.id=2381343893e74c0a919ff4d0d59b655c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:13.282128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc08b7-3688-4417-bb6c-d38a9a9f7db9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:13.318925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:13.319540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.319826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.320103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.320898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:13.321214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:13.322674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231d92-7703-4339-82c2-6d7e286012a2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:13.336552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 265/540] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:13.344288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:13.367038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:13.488688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3aa3d-d894-4b88-adb0-cf876583a4d8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:13.529965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:13.530657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.530940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.531241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.532099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=fd8aa4f4a5cb441cae26c8dafa27548c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:13.532414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:13.533888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61ae58-15b7-407e-9a2d-245cbc1b24a7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:13.540711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 276/541] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:13 2022] GET /identity/v3/role_assignments?user.id=fd8aa4f4a5cb441cae26c8dafa27548c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:13.830843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f86b-a6fe-44f7-bea9-7b4e18f411a7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.831919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f86b-a6fe-44f7-bea9-7b4e18f411a7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.832437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f86b-a6fe-44f7-bea9-7b4e18f411a7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.834394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 266/542] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:13.839992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.840791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.841268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.858843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:13.860384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:13.937666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681c42-b897-457c-9499-73bc2a731226 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.937666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681c42-b897-457c-9499-73bc2a731226 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.937666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681c42-b897-457c-9499-73bc2a731226 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.937666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 267/544] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:13.938639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 277/544] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:13.944123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:13.944479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:13.944721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:13.965730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:13.975858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:13.977062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:13.977258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.074017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.112851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905f976-f9e4-4573-9ed2-8c6e7264ee42 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.137860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 268/545] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:14.149746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.155106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.155106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.155106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.164819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:14.173529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:14.174443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:14.175739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 278/546] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:14.177715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.194210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.409907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.470176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31259-387b-4c17-85f6-f28d890a227b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.526350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.528686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.529324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.529752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.544405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:14.553428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f817-3f9e-48a6-b393-87f509cb5744 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:14.553855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:14.554844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:14.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:14.558292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 269/547] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:14.564427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.614588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.619854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.620596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.620824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.621139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.621970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:14.622427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:14.624125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:14.644912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 279/548] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:14.652237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.702587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236af3-8781-409a-9ec8-a25147b201a3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.755667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.756630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.757114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.757791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.758847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:14.759389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:14.760989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf2e2c-0bf7-44bb-99cb-f6f2521bec82 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:14.770902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 270/549] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:14.772823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e675-0c44-4c06-8a08-aea1ccbf7dbc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.773339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e675-0c44-4c06-8a08-aea1ccbf7dbc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.773815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0e675-0c44-4c06-8a08-aea1ccbf7dbc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.779981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 271/550] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:14.781367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0e675-0c44-4c06-8a08-aea1ccbf7dbc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.819514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcc26-f0b6-4888-bfad-eaf78efeca09 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.850910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.856985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.858656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.858656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.858965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.872106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:14.874047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-11bccd08-de68-4778-bd2b-f579ed51c4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:14.892616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:14.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 280/551] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.899474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.909951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0e675-0c44-4c06-8a08-aea1ccbf7dbc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:14.912358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:14.916025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:14.918046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:14.954285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:14.955494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:14.955982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:14.956566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:14.968374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:14.970467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:14.995099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 281/552] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:14.995099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:14.997641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:14.998836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:15.000281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:15.000281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:15.000281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:15.000281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:15.003344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 272/553] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:15.007798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.097321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec260e3c-e736-4fc6-b971-e502067c53cf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.110977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.133178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.133815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.134097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.134368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.135107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:15.135428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.137040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.140227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f1bf-f923-4caf-8c9b-447324b27e44 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.153579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 282/554] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:15.153579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.185147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.186842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.187385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.187868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.189078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:15.190408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.192160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.208152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 273/555] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:15.209925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.232969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.259757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb134-5639-4432-9e76-b5f87979ee7e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.295259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.296069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.296356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.296611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.307292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:15.314866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:15.316015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:15.317586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 283/556] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:15.318264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.353167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.358460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7fa60-380e-4b71-b9a7-532724061b92 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.397900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.398700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.398983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.399292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.407299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.408763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:15.424000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:15.425082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:15.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:15.430485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 274/557] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:15.431365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.453680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c797fa-397c-4c5d-b5c8-9a2abcd8673e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.493145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.493997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.494180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.494415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.503989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.505795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:15.520917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:15.521926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 284/558] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:15.523587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.535296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.573239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a9952-39cc-4ce9-8417-9b79ea662354 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.622309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.623640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.626196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.627601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.628706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.630574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:15.631554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.633508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.634010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188b135-de0b-463b-bf0e-786fcfecfc61 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.647401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 275/559] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:15.648982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.676158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.676953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.677233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.677498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.678388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:15.678746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.680169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.703362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 285/560] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:15.704242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.779525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b978a-c329-4823-a59c-e384d04ab049 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.814461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.814868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01864f3-f04a-420c-a36c-ee7b089dc7dd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.815161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.815444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.815746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.817114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:15.817432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.818902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.822354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.843263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 276/561] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:15.844123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.852482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.853298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.853583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.853857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.863931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:15.865413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5569cac6-a4d7-4563-bc53-60f44c42438d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:15.883785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:15.884710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:15.885564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:15.885564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:15.885564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:15.885564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:15.889618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 286/562] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:15.890751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb5b1-2400-4881-9a3e-50caa36ce76e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.891038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb5b1-2400-4881-9a3e-50caa36ce76e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.891336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb5b1-2400-4881-9a3e-50caa36ce76e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:15.896981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 287/563] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:15.897746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb5b1-2400-4881-9a3e-50caa36ce76e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:15.900687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:15.958793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bb393-b3ac-4442-a980-142a3515a4d0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:15.993780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:15.994544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:15.994856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:15.995162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.004682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb5b1-2400-4881-9a3e-50caa36ce76e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.040646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.041461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.041724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.041988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.052322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.053807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.056401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.064150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.065645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339b86-d237-4400-9add-ae3abd16519b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:16.071606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.072565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.073428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:16.073428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:16.073428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:16.073428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:16.073428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 288/564] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.073702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.132957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.162598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 277/565] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:16.163931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.164221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.164497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.180805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:16.181497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0380cc-4087-4939-9d86-3945b628cee5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.182024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:16.215034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.215739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.216019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.216268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.216979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:16.217284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.218691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed7bec-6d31-4e4a-8d7f-648685eec8f8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.229803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 289/566] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:16.231014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c436-3217-4191-a813-0bbe70d6a71a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.231328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c436-3217-4191-a813-0bbe70d6a71a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.231610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763c436-3217-4191-a813-0bbe70d6a71a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.235207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 290/567] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:16.236158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763c436-3217-4191-a813-0bbe70d6a71a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.249218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 278/568] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:16.249975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.275509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.340237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763c436-3217-4191-a813-0bbe70d6a71a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.350005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd16fb-4df6-4748-9ee4-1004814709e1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.353776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.376006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.376693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.376980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.377256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.377989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:16.378359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.379869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.384854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.385648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.385929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.386199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.392154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 291/569] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:16.393059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.400250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.401715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b57755ec-1085-4a24-a69f-e1c446de6e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:16.419282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.420169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.421025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:16.421025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:16.421025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:16.421025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:16.421025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 279/570] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.421945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.422234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.422512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.438880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:16.440089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:16.495713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f402-409a-4985-83c5-4c36670a1322 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.509966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 280/571] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:16.510789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.518779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.537742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.538501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.538755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.539031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.546973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.548460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:16.562925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.563885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:16.564816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 292/572] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.566073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.598059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.617888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6ace1-5995-46ea-a31f-cbb4cf379535 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.653943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.654751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.655031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.655336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.663761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e1379-30c9-4402-8986-574e751ee74c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.665422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:16.672977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.673881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:16.674726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 281/573] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.675595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.703103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.703942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.704226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.704504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.711271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.712738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:16.726292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.727208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:16.728063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 293/574] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.728530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.743788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.781224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e98f7-3938-4b0b-91aa-2592edba2f87 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.814695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.815333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.815694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.815971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.816697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:16.817007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.818401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.824248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.830395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 282/575] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:16.835178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.835272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7c04-aa7c-47e6-87b8-f29b43e31eac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.864607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.865425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.865709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.865984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.876502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:16.884419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:16.885334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:16.886199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 294/576] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:16.886878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:16.932472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066182d8-92d5-4a2e-9b88-78493204a3f5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.958808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:16.968055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:16.968732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:16.969048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:16.969322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:16.970160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:16.970469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:16.972074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:16.985911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba43d2-5ad6-4b6b-8833-cb7d64048c8e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:16.988959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 283/577] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:16.989757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.020006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.020684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.020967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.021243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.021988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.022295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.023802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.031825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 295/578] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.032481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.050467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.091380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b230a7c-6859-4878-80c0-4d5e313196bd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.123767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.124445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.124902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.125268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.126093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.126408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.127870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.129684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2adc56-a6eb-466c-b649-00def5872ae1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.139278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 284/579] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.140175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.168069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.168886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.169167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.169552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.176240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.177723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.187228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:17.191684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:17.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:17.193541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 296/580] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:17.194033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.246369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbe608-1437-4af0-950b-5c4518a52ec5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.272944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.280572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.281258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.281542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.281819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.282567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.282873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.284289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.292485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 285/581] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.293263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.302544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811b48d-15db-4972-a5c5-cd575e60ecab None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.336542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.337347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.337628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.337901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.348375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.349777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:17.367239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:17.368134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:17.369065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:17.369065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:17.369065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.369065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:17.369065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 297/582] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:17.369352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.399670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb42c-5b8a-43d8-9b0f-5d78f9e25dd9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.407112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.437491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.438660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.439127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.439419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.452075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.453838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.471327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1dd2-f65d-4cc6-895c-97fe4c754d70 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:17.472836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:17.473827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:17.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:17.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:17.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:17.474699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 286/583] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:17.474966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.502955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.511676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.512574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.512848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.513119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.526992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=15d802693dbc490c890d922d838f856c, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.528473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.541627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 298/584] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:17 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:17.542424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.585902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c32d1-6c9a-4329-9f85-be99308457cb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.625908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.626525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.626808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.627104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.627916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.628281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.630183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.649393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 287/585] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.649393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.664800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fecbf9-69eb-4cf1-885d-cdfb3834cf85 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.671952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.707595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.708500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.708861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.709211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.720010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.721588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:17.747507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:17.748915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:17.749903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:17.749903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:17.749903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:17.749903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:17.749903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 299/586] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:17.755030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.785566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.786933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74626f-afb6-401b-80ad-d183249ec639 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.824509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.825340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.825696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.825976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.826705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.827021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.828439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.841405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 288/587] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.842226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.866630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91992659-85a7-486e-a5a0-d019d1176bb8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.902746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.903378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.903669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.903945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.904674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.904986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.906440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:17.918142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 300/588] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:17.919023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:17.933445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:17.956456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ebc15-3ea7-4792-90e4-4124dd7ff578 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:17.990844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:17.991636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:17.991931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:17.992210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:17.993041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=616b28448ac548c0b9c22f1245b090b6) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:17.993354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:17.994793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.007412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 289/589] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/role_assignments?user.id=616b28448ac548c0b9c22f1245b090b6&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:18.007412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.017947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.032217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd578fab-5115-43d7-bef6-6a0a9003f768 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.088846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.089542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.089826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.090102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.090854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:18.091192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.092624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.114349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 301/590] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:18.115698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.145723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54653df-bc3e-4cb5-8147-36bedae007e0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.186361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.187214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.187513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.187780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.195214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.196697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:18.212538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:18.213438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:18.214304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:18.218473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 290/591] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:18.219386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.222701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.256036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238abd-3216-4ab1-b718-3e1d3c5bf904 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.300957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.302393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.303759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.304236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.313861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.316085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:18.336928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:18.337873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:18.338816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:18.340062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdce60-33e1-4d5f-8769-782e8536685c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.342769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 302/592] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:18.343804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.360910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.378909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.380079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.380556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.381099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.389006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.390720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:18.406005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:18.406943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:18.408216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:18.411056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 291/593] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:18.412447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.477107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62503d1-3262-48ea-98e8-f560df52c524 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.481509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.518515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.519458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.519764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.520014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.520807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:18.521116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.522576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.524584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1652e-f958-4fb2-9f74-d61c5aab2057 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.535276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 303/594] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:18.536230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.560251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.561267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.561768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.562239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.563296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:18.563799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.565437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.584662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 292/595] 10.222.0.65 () {56 vars in 1139 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:18.585793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.613921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.648068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c08eba-03bd-4975-864d-e0dffaca8773 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.685304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.685994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.686278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.686554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.687434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:18.687817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.689213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.694903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a6dd1-c9e3-4729-bc00-85a2ac3697af None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.713187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 304/596] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:18.714137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.716883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.732857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.734119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.734612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.735100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/01aa5644b3834b0d887bbfc854ebc8cd/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.755329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=01aa5644b3834b0d887bbfc854ebc8cd, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.757028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.778685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 293/597] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:18 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/01aa5644b3834b0d887bbfc854ebc8cd/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:18.780032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:18.857405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e8c81-6432-4652-8c7e-34d7707eeb31 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.897789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204912d1-71f3-40cb-bc5d-a384fd8b4046 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:18.909216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:18.923248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.923724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.927474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.927754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.946857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.950147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.951971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:18.953061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:18.954990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:18.955527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:18.981631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:18.985346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:18.999434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:19.000513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:19.001386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:19.001386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:19.001386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:19.001386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:19.003674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 305/598] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:19.011473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:19.024800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:19.026154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:19.027039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:19.027039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:19.027039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:19.027039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:19.029491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 294/599] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:19.035183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.067807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:19.161448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38971d2-ab55-4be5-9962-a60bf742b772 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.205985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264dd57-8c53-4c82-b2a9-6b74a2b62c56 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.209489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.210428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.210910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.211406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.212371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:19.212874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.214472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.240018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 306/600] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:19.248107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.283348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.284090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.284794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.285082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.285824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:19.286190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.287634 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.309627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:19.314898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 295/601] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:19.325179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.424974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282ae7-d580-46bf-8066-ba47b9631899 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.480288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.481105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.481391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.481710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.484540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:19.493888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.496767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.517185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35721c-1474-4745-8c77-bf1803c19a15 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:19.522453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:19.523506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:19.524430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 307/602] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:19.532023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.585086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.585900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.586188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.586468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.601684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.603195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:19.623058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:19.624344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:19.625240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:19.632940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 296/603] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:19.640927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.682434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:19.688126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fae88-ec0f-4fb6-9078-785b38f071eb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.735344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.736029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.736317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.737048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.737900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:19.738232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.743427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486355-a50a-4a97-b4ec-61d958d642f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.767126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 308/604] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:19.767126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649ea02-5b2c-4660-95ac-f7eead6fdc78 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.767126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649ea02-5b2c-4660-95ac-f7eead6fdc78 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.767126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649ea02-5b2c-4660-95ac-f7eead6fdc78 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.767676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3bd0d-9765-42ec-ab0f-50118a536a1c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:19.767779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 309/605] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:19.771670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.771969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.772334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.797690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:19.800030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:19.807390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:19.808119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:19.808401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:19.808681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:19.809521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:19.809831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:19.811553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:19.825612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 297/606] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:19.826421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.829301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:19.892611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 310/607] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:19.894209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:19.959396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2666-67c8-4a48-a4fa-e13d54fd5263 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.004078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.004905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.005186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.005461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/2381343893e74c0a919ff4d0d59b655c/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.008063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.022081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=2381343893e74c0a919ff4d0d59b655c, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:20.022588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4c6f4-e5cc-4930-9585-81427a919b7f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.023609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:20.042774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 298/608] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:19 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/2381343893e74c0a919ff4d0d59b655c/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:20.043840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.067121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.068501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.069109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.069601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/fd8aa4f4a5cb441cae26c8dafa27548c/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.087530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=fd8aa4f4a5cb441cae26c8dafa27548c, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:20.089475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:20.107977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.109645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 311/609] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:19 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/fd8aa4f4a5cb441cae26c8dafa27548c/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:20.173068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516bf3-ff0f-4b4c-ab0b-c0cd0eb68d0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.217469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.218403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.218656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.218936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.232503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3aecebd1-8222-45e1-9d40-ea3d568bcdd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:20.241335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:20.243440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:20.245300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 299/610] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:20.249576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.301129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035d36b-40cb-46c4-895c-98278a46f6c8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.301129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035d36b-40cb-46c4-895c-98278a46f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.301129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035d36b-40cb-46c4-895c-98278a46f6c8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.302348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 300/611] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:20.308865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.309752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.310039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.334536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:20.335842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:20.424259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 301/612] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:20.431380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d59a55-d04b-4b3c-a943-8d22887db695 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.447346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.470397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.471508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.472254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.472923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.473915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:20.474458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:20.476094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:20.485555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 312/613] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:20.497472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.504463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.527449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.610302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fc4f1-0509-48a7-9eae-10fcc51bbded None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.682056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ac32-d9d7-49cc-9a14-20d99be0158f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.707815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.709054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.709559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.710030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.721324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:20.726410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.727271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.727562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.727845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:20.731177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:20.732089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:20.732917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:20.735645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 302/614] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:20.743134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:20.743134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:20.744352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:20.775463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:20.776450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:20.777325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:20.777325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:20.777325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:20.777325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:20.779791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 313/615] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:20.785947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:20.882190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7a1e6-6bf5-42c1-a05e-422194aea1b3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:20.897171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.912835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:20.920528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:20.921449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:20.921934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:20.922398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:20.923357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:20.923983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:20.925660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:20.934741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 303/616] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:20.943245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.015609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14138-65c4-4d0d-98f3-7e8ba026bbf7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.076429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.077111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.077396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.077678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.078410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:21.078717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.082125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.103765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 314/617] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:21.109214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.126305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269905-3eb2-4784-bc2d-e1dfaf033253 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.191129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.191945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.192233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.192569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.213459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.214933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:21.247224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:21.248565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:21.249460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:21.249460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:21.249460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:21.249460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:21.250364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 304/618] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:21.256342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.282296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ff711-0698-4d8e-9d82-8072a3cec71d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.283322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.299435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.321404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.322448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.322818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.323238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.330927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.333036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0ef0a7df-025a-4bf4-b6bf-ed65ee696fac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:21.348604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:21.349776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:21.351162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:21.356172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 315/619] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:21.357667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b88c-edec-4356-b62d-95c1f4a0ca85 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.358019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b88c-edec-4356-b62d-95c1f4a0ca85 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.358370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694b88c-edec-4356-b62d-95c1f4a0ca85 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.359896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 316/620] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:21.360685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b88c-edec-4356-b62d-95c1f4a0ca85 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.385284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b67f2-c65e-401e-a36a-0b11dc24aa9a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.421164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.421835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.422119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.422396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.423140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:21.423452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.425088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0210cd6-46f9-4e71-8b07-b9506db3d897 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.438385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 305/621] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:21.441361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.441726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.442008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.460340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:21.461858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:21.478191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694b88c-edec-4356-b62d-95c1f4a0ca85 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.513660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.514344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.514648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.514902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.515775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:21.516086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.517524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.547424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 317/622] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:21.548358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.548801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 306/623] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:21.549993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.575442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.597676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.688903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94ee90-93c5-41a2-823d-82a56cac6631 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.690157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131c0ca-cc64-46c6-9ba4-e094e977813a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.725208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.725867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.726154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.726434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.733711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.733855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.734647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.735068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.735311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.735736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:21.758693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:21.759694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:21.760634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:21.763252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 318/624] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:21.764770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.816544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:21.818046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:21.826626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.881349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:21.891533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489a0c-3b7d-42db-81a0-0d472411ac44 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:21.927622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:21.928485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:21.928774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:21.929049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:21.939702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:21.945043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 307/625] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:21.946205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:21.948242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:21.949151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:21.949973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:21.954237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 319/626] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:21.955096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.053006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec898b4-7c5a-4383-80bb-86576fc2cb8d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.061437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868ee2-e1cf-425b-802b-bc8095c05fc9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.062656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.091211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.092293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.092778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.093243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.094287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.094812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.096866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.099135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.099958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.100239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.100513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.112281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.114057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.117864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 308/627] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.118970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:22.132700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:22.133719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:22.134582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:22.134582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:22.134582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.134582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:22.138569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 320/628] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:22.139448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.143524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.226621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a216-0006-4ea8-940c-44498a2ef21a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.248767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d92226-8b4f-4376-81c7-14c7c96dd802 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.263670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.264687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.265163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.265667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.266599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.267114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.268767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.282335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 309/629] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.283758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.285768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.286641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.286927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.287298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.298988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.300847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.316254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:22.325211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:22.326131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:22.326991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:22.326991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:22.326991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.326991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:22.326991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 321/630] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:22.327830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.387482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.396728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f2ec-07c1-4e82-bf13-6dade1b42379 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.432155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.432833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.433117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.433394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.434119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.434426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.434594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25a55-cc93-4ce9-a01d-48336c05328c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.435841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.448854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 310/631] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.449683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.473749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.474559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.474854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.475138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.485930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.487571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b8f134bd-40f5-4fd2-bdbc-af93ae84566a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:22.505279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:22.506226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:22.507150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:22.507150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:22.507150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:22.507150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:22.511045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 322/632] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:22.512752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbdcb9-a95b-45f4-937e-f2729284c966 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.513046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbdcb9-a95b-45f4-937e-f2729284c966 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.513321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbdcb9-a95b-45f4-937e-f2729284c966 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.514844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 323/633] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:22.515565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbdcb9-a95b-45f4-937e-f2729284c966 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.547834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.563927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e68ac-e080-4e2a-ae5c-64ceb718924d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.597955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.598555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.598838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.599140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.599861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.600176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.601558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.613090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 311/634] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.613710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.625657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbdcb9-a95b-45f4-937e-f2729284c966 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.634855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.659724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.660523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.660806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.661082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.667730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.669204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e71ca00a-4766-47e9-b7a2-2cd2b7137ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:22.683457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:22.684341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:22.685213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 324/635] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:22.686212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.686212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.686366 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.702576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:22.703793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:22.713574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe71246-dc5c-4356-82fc-8f07e253bf23 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.746477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.747181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.747469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.747746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.748822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.749129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.750516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33359af8-1811-463a-aa85-44a891975aa2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.762457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 312/636] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.763843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eda66-0f55-4d16-ad1b-654772343842 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.764133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eda66-0f55-4d16-ad1b-654772343842 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.764408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eda66-0f55-4d16-ad1b-654772343842 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.765632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 313/637] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:22.766314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eda66-0f55-4d16-ad1b-654772343842 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.776004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.778406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 325/638] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:22.779299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.874434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eda66-0f55-4d16-ad1b-654772343842 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.889675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:22.893835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7ad06-9710-4555-a9de-9736547a4267 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:22.908428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.909314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.909594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.909874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.916668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.918137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.927282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:22.927956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.928242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.928520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.929311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:22.929622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:22.931054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc75d08-0755-4ec9-b88a-6a76f1086faf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:22.931775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:22.932602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:22.933457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 314/639] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:22.933936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:22.947383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 326/640] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:22.948700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:22.948992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:22.949276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:22.965606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:22.966796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:22.997938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.032453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 327/641] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:23.033275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.034653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39e5b1-f03d-4a06-b6b8-b2d9f24b47be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.068675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.069488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.069776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.070072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.077952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.079436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:23.092961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:23.094135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:23.095134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 315/642] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:23.095730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.139160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.152392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2f071-4b60-4026-9e7f-abdadf893a7c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.185214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.186026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.186308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.186590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.194126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e875b8-6a49-4689-8dac-79eb5eeb0a9e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.196313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:23.203977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:23.204923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:23.205814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 328/643] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:23.206807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.223297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.232702 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.233367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.233647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.233918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.234704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:23.235008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.236473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.249416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 316/644] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:23.250202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.308105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e4d25-ace3-4f9f-9b55-124ad43ad2a8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.340116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.340769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.341081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.341348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.342172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:23.342482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.343940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.349158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6996-3313-4ab8-a6d6-3b64946482ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.349838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 329/645] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:23.350586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.365956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.389045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.390364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.390797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.391269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.405198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:23.414267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:23.415508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:23.416533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:23.418752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 317/646] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:23.419929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.469770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d2a3d-6c7f-480d-a32a-df19f47b9351 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.474037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.506781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.507516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.507918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.508205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.509059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:23.509381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.510982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.530871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 330/647] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:23.531699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.536016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0eca-8464-4a92-aab1-c7a60e58d199 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.576438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.576650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.576889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.577167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.578642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:23.578642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.580102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.589797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 318/648] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:23.590641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.609919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.641528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92111ad-579e-47fa-a954-0fd4b6888fdb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.675452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.676363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.676642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.676903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/616b28448ac548c0b9c22f1245b090b6/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.691578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=616b28448ac548c0b9c22f1245b090b6, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.693065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.694620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d13-2520-4e29-96f2-40b7ad0bc9f4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.704696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.711116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 331/649] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:23 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/616b28448ac548c0b9c22f1245b090b6/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:23.711116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.759440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.760546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.761858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.762357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.767488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:23.768151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.770002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.785311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 319/650] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:23.786642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:23.839850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354ed09-b6d1-4622-8a19-613ab25f5bdd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.863395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:23.883906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.884592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.884880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.885162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.901230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba38070-982b-465d-a47b-4cf9047928ca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:23.944377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:23.945444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:23.946124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:23.946406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:23.955655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.957259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.961190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:23.962717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:23.985064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:23.986047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:23.986931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:23.986931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:23.986931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:23.986931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:23.986931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 320/651] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:23.992225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.026235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.077349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 332/652] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:24.078338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.101483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.127461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc81d76-688d-4c71-abb9-323b85be0cda None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.163937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.165162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.165649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.166253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.178502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.180515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.185709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6be82-8334-4fbc-be1a-4aca70b5d623 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:24.200512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:24.201880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:24.203646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:24.203646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:24.203646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:24.203646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:24.211340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 321/653] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:24.212636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.226319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.227340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.227823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.228330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.229282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:24.229784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.231410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.243434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 333/654] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:24.244588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.299731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.326123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.327992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eede9-a0dd-4abc-8130-ae64e68ff34a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.346251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbf0ff-d18c-404d-92cc-bf643018c812 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.365716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.366522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.366802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.367092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.379813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.381289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.382939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.383876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.384354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.384840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.385775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:24.386287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.387888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:24.399224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:24.400123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:24.400988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:24.400988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:24.400988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:24.400988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:24.403918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 334/655] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:24.405027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.405874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 322/656] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:24.406731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.510157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84844cf-ebc0-4719-87d2-cd1cf5ba39cc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.525093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e526ebb-6bc1-42f9-8577-2c22e3fb6ae6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.552055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.552928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.553208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.553481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.561713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.562085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.564733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.570438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.573604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.574280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.574558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.574833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.575754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:24.576880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.578278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:24.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:24.588202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 09:29:24.589337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:24.592130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 335/657] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:24.593053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.604378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 323/658] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:24.604378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.728617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ec785-34b0-4452-bf38-40d407b6a9d4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.734259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f32a-8d73-4fb3-b2a3-3ae81edd5d77 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.770031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.770996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.771508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.771974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.773247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:24.773801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.775504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.777910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.779270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.779270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.779270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.786859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.788412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.795759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 324/659] 10.222.0.65 () {56 vars in 1139 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:24.796836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:24.807761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:24.808757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:29:24.809708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:24.813537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 336/660] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:24.814295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:24.850345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.875421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:24.915816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c13c32-4ee1-457d-92f2-4fc84d5aebdb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.919539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1ee34-6d8c-432e-bfad-f61f6255dd41 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:24.954019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.954268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:24.954708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.954993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.955309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.956167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:24.956650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:24.957191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:24.965301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:24.967158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:24.982670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:24.983733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:24.984643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:24.991788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 325/661] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:24.992972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.024782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.026249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.084129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.105312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b962a-4b67-44e2-9640-12949e40f779 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.108273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.132413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 337/662] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:25.133369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.145961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.147040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.147652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.148197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.149340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:25.149853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.151674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.171526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 326/663] 10.222.0.65 () {56 vars in 1137 bytes} [Fri Jun 24 09:29:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:25.172723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.239067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c49cb-6427-4060-bbc9-c872026c2069 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.273563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.274239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.274519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.274863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.275610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:25.275939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.277369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.284558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff15f84-ccb3-4611-9ec5-39563649144b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.294204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 338/664] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:25.294204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.317263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.327552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.329264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.330152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.331132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.342848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.344628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:25.364505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:25.365485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:25.366383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:25.366383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:25.366383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:25.366383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:25.377595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 327/665] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:25.377595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.396506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.408805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4b3e8-a49b-40ae-9885-b900adfd9e85 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.454673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.455631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.456169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.456449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.495418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3744a-c675-4d37-9089-987971b77255 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.530199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.531241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.531641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541cc3c-4bb4-4ab2-a6c8-97ad1a87d484 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.531964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.532252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.532532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.533428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=01aa5644b3834b0d887bbfc854ebc8cd) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:25.533752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.535255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.551472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 328/666] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/role_assignments?user.id=01aa5644b3834b0d887bbfc854ebc8cd&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:25.552288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.553755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.657478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.661604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 339/667] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:25.663254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816249-7c79-46f1-a9ca-1177f3e1fb1e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.663565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816249-7c79-46f1-a9ca-1177f3e1fb1e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.663847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816249-7c79-46f1-a9ca-1177f3e1fb1e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.665446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 340/668] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:25.669145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816249-7c79-46f1-a9ca-1177f3e1fb1e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.693301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a7284-b5ba-48a0-8384-5e76b82aa414 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.743976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.744664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.744958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.745226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.746085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:25.746507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.748722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987aee1-6d61-4507-bc13-824832f83f83 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.762614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 329/669] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:25.768234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.768234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.768234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.788050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:25.789163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:25.812269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816249-7c79-46f1-a9ca-1177f3e1fb1e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:25.835551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.855495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:25.856363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.857034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.857034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.868574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 330/670] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:25.869934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.875038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:25.876570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-bb40b91c-6d72-42ad-b81b-8c444a2d4498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:25.898071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:25.899199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:25.900170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:25.900170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:25.900170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:25.900170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:25.900170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 341/671] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:25.900858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468c99-802c-47c5-99aa-ec5ed148ce43 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:25.901143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468c99-802c-47c5-99aa-ec5ed148ce43 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:25.901421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24468c99-802c-47c5-99aa-ec5ed148ce43 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:25.902836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 342/672] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:25.907335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24468c99-802c-47c5-99aa-ec5ed148ce43 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:25.943953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:25.981656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34776f2f-7d16-4957-94ee-88d563c90945 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.013947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24468c99-802c-47c5-99aa-ec5ed148ce43 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.020483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.021301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.021583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.021860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.028944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.030411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:26.045710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:26.046600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:26.047599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 331/673] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:26.048070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.055706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.056522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.056803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.057077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.067665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a0055e13-63de-4cb0-9470-2cb25bee10aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:26.075179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:26.076136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:26.076954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 343/674] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:26.077647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.078339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.078339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.095739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:26.097208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:26.123700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.165867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6d5fd-beaa-4747-ab55-15f41ed7dbc6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.168179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 344/675] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:26.176934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.194972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.204472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.205153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.205442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.205721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.206513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.206821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.208243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.220192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 332/676] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.220991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.288720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aac0a0-d226-4b51-a55e-6a2da5cfc0cc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.326090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90a338-b10f-4af6-b64b-f8f63b79c8fb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.326288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.327218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.327747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.328325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.329139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.329438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.330957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.348802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 345/677] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.348802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.367397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.368157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.368451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.368728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.369484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.369792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.371206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.375409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.381010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 333/678] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.381658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.437315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.466843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aa80f-cebd-44f3-9ece-3773a22ae6db None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.492857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8197aa-795c-445e-9bc9-08a460a52581 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.500738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.501724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.502007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.502284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.513568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:26.521263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:26.522193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:26.523014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 346/679] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:26.523739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.536315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.537509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.537806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.538121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.545873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.547370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:26.562682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:26.563734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 334/680] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:26.569978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.614056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.634293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078c1b-5692-40bf-b015-ffb0c2bb8ac0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.669950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.670791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.671108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.671379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.672241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=2381343893e74c0a919ff4d0d59b655c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.672559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.673645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337b1b-bb5a-47fd-95e3-37c594891a5f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.674023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.680259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 347/681] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/role_assignments?user.id=2381343893e74c0a919ff4d0d59b655c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.691576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.696485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.730438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.731441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.731796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.732144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.751435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.752991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:26.776351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:26.777358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:26.778243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:26.778243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:26.778243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:26.778243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:26.778243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 335/682] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:26.778976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.836049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e8d2d-c443-4f73-9c8c-ae3b3a5abc83 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.886814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.888531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.888822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.889114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.890583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.890583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.892311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.896959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cbfde-f521-46fe-8758-a3d6b159d435 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:26.901416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 348/683] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.902257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:26.917620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:26.944738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:26.944738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:26.944738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:26.945096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:26.945782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:26.946087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:26.949221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:26.970989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 336/684] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:26.972153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.007237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.029304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648406e7-f4ad-45b6-824d-9c6d17bed6dd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.071387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.072066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.072346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.072618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.073333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:27.073635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.075095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.093287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 349/685] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:27.094201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.107350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7757e1-c322-4f39-8652-89412dbb1789 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.154667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.155997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.156836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.157524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.164695 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.173227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.175026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:27.196936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:27.198079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:27.199163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:27.199163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:27.199163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:27.199163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:27.202778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 337/686] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:27.203642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.210016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a84a6-780a-4647-93cf-a3d66dcd6949 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.260112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.261400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.261833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.262256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.263296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=fd8aa4f4a5cb441cae26c8dafa27548c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:27.264298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.266175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.275862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 350/687] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/role_assignments?user.id=fd8aa4f4a5cb441cae26c8dafa27548c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:27.276904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.310443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.383191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768380b-37e9-44ae-83a7-1a2806f09fa3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.401668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d182222-eab6-4181-bab6-542ab45a8d63 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.438729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.444945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.445882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.446515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.446970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.447983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:27.448461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.450924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.465833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 351/688] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:27.476681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.490477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.493033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.493033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.493033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.505343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.506839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:27.532729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:27.533981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:27.534972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:27.543491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 338/689] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:27.544729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.642569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcad0fe-6121-4679-85e1-8a30258240d9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.683282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.684021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e006d9c-a66d-4178-b573-fc639bebfa54 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.684350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.684720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.685082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.694423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.695989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:27.711906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:27.713086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:27.714266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:27.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 352/690] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:27.722486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.727909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.728807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.729239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.729664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.730656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:27.731132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.732731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b652d-66aa-482a-a698-3b5fd34c2259 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.735319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.759732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 339/691] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:27.760729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb577b-c076-43f2-b5b9-6577e51323d9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.761064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb577b-c076-43f2-b5b9-6577e51323d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.761303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb577b-c076-43f2-b5b9-6577e51323d9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.762925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 340/692] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:27.769402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb577b-c076-43f2-b5b9-6577e51323d9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:27.795893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:27.853895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bf885-8124-41ae-9b18-4dd9344cb6e4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.889956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb577b-c076-43f2-b5b9-6577e51323d9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:27.900331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.901667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.902418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.902866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.904227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:27.905156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:27.907317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660198d5-51a7-4256-b2d3-5c3bb77ebacf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:27.926073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:27.926751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.927026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.927334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.938068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 353/693] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:27.940521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:27.941664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:27.942164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:27.966231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:27.967823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:27.991699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.002732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.004258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.057163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 354/694] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:28.058055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.110888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.117413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 341/695] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 09:29:28.118703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.193226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92df540-74a1-46f0-b028-0a882c9eca01 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.229978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.241632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cecb1-ecd5-48fc-8345-06e62381a050 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.243953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.245504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.245786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.246063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.280493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.281787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.282266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.282755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.293971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.295870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:28.314876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:28.316034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:28.316916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:28.316916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:28.316916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.316916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:28.319002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 342/696] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:28.320350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.328249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.329728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.385176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.444431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050059a-fe94-4d4a-8f28-2ed15091d71c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.471828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 355/697] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:28.471828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.498603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.499965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.500525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.501019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.513360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.515284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.519645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:28.538332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:28.539323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:28.540226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:28.540226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:28.540226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.540226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:28.542628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 343/698] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:28.543889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.588596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e087e-6186-4618-8c7b-04252bd24e30 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.640563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.641239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.641518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.641789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.642507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:28.642809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.645444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.656841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.659113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaffd6-86ec-428a-b1a6-734f253fa074 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.660895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 356/699] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:28.662075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.701791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.703061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.703666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.704219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.716586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.718470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:28.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:28.740328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:28.741226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:28.741226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:28.741226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:28.741226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:28.749557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 344/700] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:28.750912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.796128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402b0f-2113-490c-8684-a5d105406d66 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.818237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.833541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.834582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.835101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.835591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.836515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:28.837142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.838822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.853743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 357/701] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:28.854988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.882096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b873a6-0952-4535-b61d-4fd5fc07345d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:28.919303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:28.930005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:28.930842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:28.931262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:28.931541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:28.932249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:28.934623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:28.964123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 345/702] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:29:28 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:28.964766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:28.974768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbecaf-1972-4cb3-9426-61dba00d281b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.030068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.031773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.032563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.033058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.034362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.034916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.037378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26229f70-c8df-40c1-adbf-b646fefc9d18 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.062890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 358/703] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:29.062890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f881d-0aba-4fa1-8f4b-91e948c4af10 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.062890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f881d-0aba-4fa1-8f4b-91e948c4af10 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.062890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f881d-0aba-4fa1-8f4b-91e948c4af10 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.065220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 359/704] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:29.066660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f881d-0aba-4fa1-8f4b-91e948c4af10 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.093336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f17835-5a8b-42d2-b509-08581cf4b8cf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.100796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.144665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.145483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.145771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.146044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.153952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.155502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0dc34131-9938-45fc-b456-262c598cb946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:29.172044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:29.173173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:29.174314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:29.176700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 346/705] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:29.178325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.178886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.179333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.196685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:29.198137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:29.201868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f881d-0aba-4fa1-8f4b-91e948c4af10 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.227317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.249005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.250021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.250357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.250993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.259473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.261033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:29.276113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:29.277095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:29.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:29.278393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 347/706] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:29.278393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.282382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 360/707] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:29.283554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.399151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.408937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a35a71-4bd3-42b3-a89d-f7fab319f4ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.415627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874a0f-1c49-484f-8bab-3131d13395c8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.451364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.452638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.452755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.453178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.453579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.453759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.453860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.454134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.454936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.455114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.455274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.455767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.456962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.457583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.476752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 361/709] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:29.476752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.477416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 348/709] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:29.478489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.526166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.593627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746b078-ac1f-4e7d-9e58-728d026f9791 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.609505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660ffae-3d02-4480-95c9-0e68813af51c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.629256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.630081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.630412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.630629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] PATH_INFO: `/v3/services/network` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.631331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.632729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:29.650412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:29.651570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:29.652429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:29.654313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410442e-e845-4a07-8925-30191412860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.655224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410442e-e845-4a07-8925-30191412860b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.655678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410442e-e845-4a07-8925-30191412860b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.656122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410442e-e845-4a07-8925-30191412860b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.656956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 362/710] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:29.657309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410442e-e845-4a07-8925-30191412860b None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.657810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410442e-e845-4a07-8925-30191412860b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.657919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.659703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410442e-e845-4a07-8925-30191412860b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.673522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 349/711] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:29.673522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410442e-e845-4a07-8925-30191412860b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:29.727505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.803198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387fda6-955c-4bef-9f60-ec00f0e6763d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.807234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:29.832579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410442e-e845-4a07-8925-30191412860b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:29.846212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.847332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.847332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.847501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.848296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=616b28448ac548c0b9c22f1245b090b6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.848698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.850102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753837eb-c0c2-4ae6-97b9-4a134ae9daa5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.856655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 363/712] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/role_assignments?user.id=616b28448ac548c0b9c22f1245b090b6&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:29.857937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51e02-b10b-475d-9c8e-6af41ee6cef6 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.858228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51e02-b10b-475d-9c8e-6af41ee6cef6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.858512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51e02-b10b-475d-9c8e-6af41ee6cef6 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.864446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 364/713] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:29.865982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.866275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.866555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.884676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:29.885919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:29.905048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:29.905798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:29.906088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:29.906366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:29.907106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:29.907426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:29.908997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:29.922463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 350/714] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:29.929399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:30.000764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 365/715] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:30.061584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:30.071444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:30.083961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ef06-d642-43ab-a2e8-d29244d55a63 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:30.125813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:30.126421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:30.126709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:30.126991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:30.127884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:30.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:30.129598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:30.140751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 351/716] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:30.194482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:30.199611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f412d7-af43-497f-8f7a-ffc94f2550c9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:30.241150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:30.242067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:30.242344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:30.242612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:30.243313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:30.244729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:30.258966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:30.260030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:30.260865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 366/717] 10.222.0.65 () {56 vars in 1129 bytes} [Fri Jun 24 09:29:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:30.266491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:30.389856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db04eba-2eb3-4e3d-ad04-e9749f1da827 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:30.424584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:30.425624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:30.425916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:30.426196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:30.426915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:30.427259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:30.428659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fd702-39f3-479d-8f88-2004f76a254c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:30.434948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 352/718] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:30.440256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:30.583770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98f512-aa75-4927-8560-b56017d371a0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:30.629126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:30.629811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:30.630101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:30.630372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:30.631109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:30.631418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:30.632810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49203d73-93d3-4af5-a37f-3e7656640a60 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:30.644423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 367/719] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:31.345762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9cfa-b4b0-4445-b10f-0bb8b5c3c64d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:31.346328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9cfa-b4b0-4445-b10f-0bb8b5c3c64d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:31.346412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9cfa-b4b0-4445-b10f-0bb8b5c3c64d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:31.348376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 353/720] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:31.353478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:31.354072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:31.354567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:31.380055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:31.381703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:31.411068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:31.478526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 368/721] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:31.511578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b9cfa-b4b0-4445-b10f-0bb8b5c3c64d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:31.530717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:31.648204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b9cfa-b4b0-4445-b10f-0bb8b5c3c64d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:31.697830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:31.698721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:31.699012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:31.699331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:31.719920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:31.728017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:31.728931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:31.729752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:31.730377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 354/722] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:31.735010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:31.884616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d67669-ca5a-406c-9037-2699d625c51d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:31.932796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:31.933769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:31.934189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:31.934599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:31.935591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:31.936031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:31.937557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:31.946156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 369/723] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:31.951570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.050392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.087161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7950ea9-6ae7-41d9-870c-2055253c9d87 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.132920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.133740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.134027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.134306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.148553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:32.150028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-883a3954-7d1f-401f-b05b-3341889bc592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:32.173907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:32.175372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:32.176250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:32.176250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:32.176250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:32.176250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:32.176250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 355/724] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:32.180787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.194619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3365-5e28-4a05-8c65-455227724dbc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.194896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3365-5e28-4a05-8c65-455227724dbc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.195146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3365-5e28-4a05-8c65-455227724dbc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.196831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 356/725] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:32.203160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.203160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.203160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.222035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:32.223287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:32.227431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.333401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 357/726] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 09:29:32.356523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.362966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5bfe3-1548-411a-b5e3-b293ad1d6e3e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.432913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.433618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.433897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.434171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.434919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:32.435254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:32.436692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:32.453143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.459397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 370/727] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:32.464532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.504445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41508d0-7682-43ce-82d7-93c416f63a7e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.539930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.540734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.541016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.541291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.551903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:32.559371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:32.560299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:32.561408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 358/728] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:32.566644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.570534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.618152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1cf6a-1cdf-4ed0-8519-0824e39d2efa None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.663632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.665135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.666078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.666685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.677370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:32.680082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:32.684408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d695d-e6c5-4a79-bf49-3cabf2230fb5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.699524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7f35702d-f2d0-468a-b04b-315636062db3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:32.705386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:32.706341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:32.707310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:32.711265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 371/729] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:32.713795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4c198-a7d7-43dd-989b-527cdf8e9b03 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.714311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4c198-a7d7-43dd-989b-527cdf8e9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.715095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4c198-a7d7-43dd-989b-527cdf8e9b03 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.721702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 372/730] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:32.723026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4c198-a7d7-43dd-989b-527cdf8e9b03 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.729418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.730379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.730866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.731404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.732883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:32.733413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:32.735997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03753fc1-efec-47f5-9f89-b4299a3e94a0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:32.750989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 359/731] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:32.752468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297c84-e2de-4043-9163-c786483ffa46 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.752777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297c84-e2de-4043-9163-c786483ffa46 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.753062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297c84-e2de-4043-9163-c786483ffa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.770845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68297c84-e2de-4043-9163-c786483ffa46 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:32.772088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68297c84-e2de-4043-9163-c786483ffa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:32.833200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4c198-a7d7-43dd-989b-527cdf8e9b03 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.843479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 360/732] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:32.844674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94399bd1-7160-474a-93d2-85f6dc1dc2cb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.844962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94399bd1-7160-474a-93d2-85f6dc1dc2cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.845237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94399bd1-7160-474a-93d2-85f6dc1dc2cb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.846729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 361/733] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:32.847258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94399bd1-7160-474a-93d2-85f6dc1dc2cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:32.885485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.886179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.886466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.886800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.887683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:32.888044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:32.889503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09015-e2f2-4a7b-96a3-62e89798fdba None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:32.898353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.914880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 373/734] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:32.916585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.916820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.917100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:32.933646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:32.934865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:32.954291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94399bd1-7160-474a-93d2-85f6dc1dc2cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:32.957343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:32.989126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:32.989929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:32.990210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:32.990479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.000817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.002318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.019448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 374/735] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:33.019949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:33.020839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:33.021815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:33.021815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:33.021815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:33.021815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:33.021815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 362/736] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:33.022507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.023464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.126725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78bc5-8e60-478c-b7f2-10be208db63c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.134840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630dca7-ac21-4af9-b26a-f4ec8445a340 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.162328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.164206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.164931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.165120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.165456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.172170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.172928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.173220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.173493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.174243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.175730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.192557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 375/737] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:33 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:33.200397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.317888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.350704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c06f4-bcc8-487c-b35f-d8176545407a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.370102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.372875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.396713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.397649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.398080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.398502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.399441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:33.399757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.401167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.415107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 376/738] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:33.416572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.472963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.491813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 363/739] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 09:29:33.492667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.539792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ac73-06c9-4c79-a7b8-c9f01c2572d7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.553455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.576267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.577004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.577634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.577915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.588517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-79d9424b-03cd-4c21-be70-785b645c7fbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:33.596433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:33.597367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:33.598211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 377/740] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:33.599400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956d2f-5979-4a48-9603-00de537e5dbb None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.599562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956d2f-5979-4a48-9603-00de537e5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.599848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956d2f-5979-4a48-9603-00de537e5dbb None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.601252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 378/741] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:33.605815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b956d2f-5979-4a48-9603-00de537e5dbb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.606910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60733-b867-4850-b382-f370964f52ec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.645187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.645993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.646277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.646553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.658988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.660811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:33.679262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:33.680292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:33.681225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 364/742] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:33.681821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.718192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b956d2f-5979-4a48-9603-00de537e5dbb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.720030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.761939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.763980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.764467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.764939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.780059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.782829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.790162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946aa2e9-714e-4db4-a8d1-a3186e61c7fa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-662f183a-5cc0-4e06-86e8-ed817f6fbcdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:33.802199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:33.803159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:33.804034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:33.804034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:33.804034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:33.804034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:33.804034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 379/743] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:33.804776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31d51-1545-447b-b440-1abf989079ce None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.804972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31d51-1545-447b-b440-1abf989079ce None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.805364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31d51-1545-447b-b440-1abf989079ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.813243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:33.821978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-12f31d51-1545-447b-b440-1abf989079ce None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:33.823192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12f31d51-1545-447b-b440-1abf989079ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:33.834603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:33.835279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:33.835570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:33.835850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:33.836605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:33.836912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:33.838281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:33.847464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 365/744] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:33.848243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.896623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 380/745] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:33 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 09:29:33.898026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31d51-1545-447b-b440-1abf989079ce None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:33.969489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d149f97-dea6-468b-92ce-70bf08b4a8ec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:33.986063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.007988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31d51-1545-447b-b440-1abf989079ce None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.010781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.011736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.012147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.012546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.013738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:34.014046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.015512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.037417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 366/746] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:34.038430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.051812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.052494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.052774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.053041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.053764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:34.054067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.055484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.071654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 381/747] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:34.071654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.085480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.156311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f7ab-f6f7-479a-b98e-2b38038e823e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.170414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b924a3-6808-4e73-8b32-b332c726a826 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.190540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.191448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.191801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.192117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.202439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.203969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.208058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.208804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.209145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.209486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.210267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.211840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:34.221856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:34.222917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:34.224161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:34.224161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:34.224161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:34.224161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:34.227816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 367/748] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:34.228655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.228762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 382/749] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:34 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:34.229589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.235980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.378922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb3665-a7af-4ab8-9557-415080aa567d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.386503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.405269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407e786-3b1a-48ba-9cdc-acec340f057d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.433128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.434193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.434662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.434936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.442939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.444499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.448728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.449547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.449827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.450098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:34.461064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:34.462074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:34.463022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 383/750] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:34.464545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.469923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.472147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:34.500452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:34.501406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:34.502331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:34.502331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:34.502331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:34.502331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:34.504965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 368/751] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:34.504965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.533428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.583471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb4c8-71fb-4da2-aa0a-3e85e3961627 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.625982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.626653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.626933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.627297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.628024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:34.628335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.629586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36e34d-7948-4629-b9e4-0ddc5b7b0279 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.629699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.646491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 384/752] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:34.646491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.676484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.677201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.677489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.677820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.678557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:34.678864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.680617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.692040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.696325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 369/753] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:34.697274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.780666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abb12-9a09-4e6e-ad08-0ac2da5e01db None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.808900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:34.812291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f5a2-ac13-4107-a51b-ba2d0394f186 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.828711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.829803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.830185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.830597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.831754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:34.832168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.834088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.850544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 385/754] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:34.852562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.857598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:34.858410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:34.858691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:34.858962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:34.870029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:34.872766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:34.893295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:34.894499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:34.895833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:34.899121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 370/755] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:34.900120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:34.973514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee3103-837f-4ccf-a589-f1bd02a7cb3f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:34.980073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.013912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.014764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.015173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.015548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.016485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=faa76f7a5fa04d9ca1d2489cc12e9e47) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:35.016917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.018535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.031110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 386/756] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/role_assignments?user.id=faa76f7a5fa04d9ca1d2489cc12e9e47&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:35.031110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.056002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd782-0152-41ff-bb5f-11288c4c4564 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.129278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.143603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.144355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.144638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.145027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.158252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.159913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:35.188973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:35.190136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:35.191047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 371/757] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:35.200157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.234616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484047f-58ed-41f2-a355-acffd38cc484 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.281707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.282398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.282683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.282961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.283944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:35.284253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.285814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.317800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 387/758] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:35.323361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.340419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c3c95-948d-4854-a8ec-819cdb502075 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.375534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.387828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.389776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.390537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.390979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.392247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:35.393267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.395299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844dabc-1647-4aef-8c61-8269e6301b6c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.412795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 372/759] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:35.418467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ee84-952f-4e47-9e73-7544f294f5db None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.418694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ee84-952f-4e47-9e73-7544f294f5db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.419005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ee84-952f-4e47-9e73-7544f294f5db None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.420403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 373/760] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:35.421954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564ee84-952f-4e47-9e73-7544f294f5db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.450955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cecc-fdf5-4ff4-94c5-e2835f89fa93 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.452558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.491229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.492044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.492442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.492778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.549300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564ee84-952f-4e47-9e73-7544f294f5db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.564781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.567137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.588342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.589338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.589624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.589901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.592119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=15d802693dbc490c890d922d838f856c, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.593642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa65be-17eb-4fd0-bf54-d2f753c54518 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.612084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 374/761] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:35 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:35.613703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.614000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.614532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.638178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:35.639399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:35.678945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.699997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 388/762] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:29:35.704164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.707401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:35.730814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 375/763] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:35.777030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.837293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b8771-c097-4534-a7b2-510537f50cb6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.882827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.885614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.885614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.885614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.904147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.906289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:35.936020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:35.936960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:35.937864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:35.937864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:35.937864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:35.937864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:35.937864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 389/764] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:35.944087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:35.949449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051639f-ca44-473c-b30d-4fd9a8857f04 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:35.986783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:35.987647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:35.987935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:35.988277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] PATH_INFO: `/v3/services/network` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:35.988939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:35.990331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:36.004509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:36.005534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:36.006363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 376/765] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:36.014456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.016066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.051398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.087815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71207361-9c20-4f96-929a-faa517c67a61 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.120686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c31b9-ebbc-4f1a-a94a-63f3fe6980ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.140738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.141433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.141717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.141995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.142780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.143579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.145147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.162288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 390/766] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:36.164451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.165141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.165427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.165704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.166425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.166766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.169064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.178473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.185762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 377/767] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:36.191465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.323421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abac34-07a8-4bd0-a015-8aba56f9e22d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.337879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.360246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.361137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.361425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.361737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.368669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.369636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.371311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.373531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cc921-af35-4c0f-8e72-8d741449337e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:36.388042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:36.389195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:36.390293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:36.391533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 391/768] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:36.403532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.457726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.458274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.458528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.458813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.459631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.462353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.462353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.467766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 378/769] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:36.490385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.528818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935344c0-605e-49fc-8cf1-68d1362d0d80 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.573574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.574831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.575561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.576018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.577661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.578162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.580106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da961-7194-4ee6-b4d7-3169e593dc62 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.601353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 392/770] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:36.601353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffeb60-8f50-4b91-8b47-39eb73918965 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.601353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffeb60-8f50-4b91-8b47-39eb73918965 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.601353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffeb60-8f50-4b91-8b47-39eb73918965 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.602396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 393/771] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:36.607066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffeb60-8f50-4b91-8b47-39eb73918965 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.626099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eeb7fd-2388-44c9-8f65-29aae3d5c02c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.659427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.665279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.666015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.666353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.666579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.667730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] RBAC: Adding query filter params (interface=public, service_id=fba89d1a8140474180745ce444bc7e9d, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.667843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.669270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad356-ca7a-4521-95e2-743624162ba1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.683329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 379/772] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/endpoints?service_id=fba89d1a8140474180745ce444bc7e9d&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:36.683329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.683329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.683329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.700620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:36.701818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:36.723727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:36.741134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffeb60-8f50-4b91-8b47-39eb73918965 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.791096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 380/773] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:36.791096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.801835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.802318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.802606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.802885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.804981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=616b28448ac548c0b9c22f1245b090b6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:36.806011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.809156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.817624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 394/774] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/role_assignments?user.id=616b28448ac548c0b9c22f1245b090b6&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:36.818723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:36.908574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc64d-0e3a-4624-9f0a-3c7cfca162af None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:36.948765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:36.949879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:36.950368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:36.950870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:36.951831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:36.953464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:36.960385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 381/775] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:37.004810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.009542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958d109-4006-4ba3-a6d0-ba16d3bda97c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.061272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.062944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.063464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.063939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.064919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.066608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:37.084405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:37.085405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:37.086241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:37.088876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 395/776] 10.222.0.65 () {56 vars in 1129 bytes} [Fri Jun 24 09:29:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:37.095281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.121641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.213011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bf048-c70d-49fe-b526-59f117e2863a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.249752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.250715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.251194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.251617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.252742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:37.253226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.254767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c934f-f70c-4981-a6d3-a9d2c268f2bd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:37.260298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 382/777] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:37.266084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.374849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2e555-451d-4575-a200-7c2b63d890a0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.404497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf83d0-5308-4212-b356-9ee4233f98f1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.404788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf83d0-5308-4212-b356-9ee4233f98f1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.405075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf83d0-5308-4212-b356-9ee4233f98f1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.406712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 383/778] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:37.411093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.412048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.412529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.413086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.413195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.413373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.413655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.414466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:37.414976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.416673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:37.424426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 396/779] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:37.434118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:37.435967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:37.436404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.503337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.513680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 384/780] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:37.544522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.563344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.618863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d29b3f-0bf7-40fd-a622-5bc4b0d9bfe9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.653178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7194c9-a023-495a-abe7-cb256c1a75af None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.683690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.684391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.684680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.684963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.685848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] RBAC: Adding query filter params (interface=public, service_id=9f63dcc6e39e4cf383f192d9fbaaae65, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:37.686164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.687925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:37.691527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a44db-17c5-4005-9020-086991b60565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.692586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44db-17c5-4005-9020-086991b60565 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.693089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44db-17c5-4005-9020-086991b60565 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.693614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44db-17c5-4005-9020-086991b60565 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.703267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 397/781] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/endpoints?service_id=9f63dcc6e39e4cf383f192d9fbaaae65&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:37.708818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.710637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a44db-17c5-4005-9020-086991b60565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-209a44db-17c5-4005-9020-086991b60565 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:37.726971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:37.727982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:37.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:37.735811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 385/782] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:37.739441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a44db-17c5-4005-9020-086991b60565 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.863581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.876520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a44db-17c5-4005-9020-086991b60565 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.911758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:37.915295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.916304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.916808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.917313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.918268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:37.918774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.920423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:37.929940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c96777-3803-4fc4-8008-32a19da669ad None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:37.931238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 386/783] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:37.941521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:37.984815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:37.985450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:37.985737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:37.986015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:37.986811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:37.988258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce798a1-16a1-4f92-b4a4-a59265b26f27 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.000483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 398/784] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:38.064882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537a49-9e60-42a7-bbf5-7644bc3f9815 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.074262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942f00-c6e9-41f9-a5fb-d191d1848dea None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.074754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942f00-c6e9-41f9-a5fb-d191d1848dea None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.074832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7942f00-c6e9-41f9-a5fb-d191d1848dea None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.077758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 399/785] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:38.082670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.083003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.083273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.109823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.111068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.111715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.112224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.114058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:38.118761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:38.119434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:38.137794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:38.142258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:38.173197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:38.174466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:38.175582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:38.175582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:38.175582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:38.175582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:38.181819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 387/786] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:38.188410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.247243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 400/787] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:38.267635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.311907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feae01-7376-4b70-9a44-322270d4ca99 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.348624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.349162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.349548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.349929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.350784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:38.351338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:38.353031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.359781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:38.369085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 388/788] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:38.374581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.381823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d72da-b103-4203-a415-d9c54704d135 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.431598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:38.431832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.432655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.433028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.433213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.460154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:38.468008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:38.469313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:38.470381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:38.478719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 401/789] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:38.483801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.499336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa229b-e3bb-403e-9207-64743e445dc1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.538604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.540598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.541045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.541461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.550438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:38.552164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:38.573811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:38.574968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:38.576227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:38.580240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 389/790] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:38.588063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.679830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b562181-b777-46e5-bcdb-81e6c9e111d6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.771372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95a54-2974-404c-b186-d39618a3cf0c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:38.775793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.776508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.776798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.777080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.777850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:38.778160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:38.779601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.787472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:38.805566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 402/791] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:38.808026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:38.808713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:38.808997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:38.809274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:38.810122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:38.810434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:38.811901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:38.812630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.828408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:38.831368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 390/792] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:38.836911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:38.982000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2e1b9-f31b-4c04-944c-b76e0174ae66 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.001606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3346196-46e1-410c-9956-827429d94ce5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.045357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.046180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.046466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.046745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.064376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.065897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.067854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.068678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.068964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.069242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.086877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.088390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:39.098534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:39.099785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:39.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:39.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:39.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:39.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:39.100989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 403/793] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:39.112254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:39.116005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:39.116891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:39.117933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:39.117933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:39.117933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:39.117933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:39.117933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 391/794] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:39.128280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.165569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.181546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.259197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8239d-dd94-46c9-a139-ee12ccefd481 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.268443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9f6a7-0113-4c44-ab1e-f55f8fa6d6f2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.294021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.294699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.294989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.295296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.296019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:39.296333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.297718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.309921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 392/795] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:39.315379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.341571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.342258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.342543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.342825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.343588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:39.343902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.345318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04045-3b2a-458a-a5cc-6b4ac8f23933 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.369584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 404/796] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:39.371051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d902cb-0c0d-4af8-9c62-cb191f838748 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.371387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d902cb-0c0d-4af8-9c62-cb191f838748 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.371665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d902cb-0c0d-4af8-9c62-cb191f838748 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.377614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 405/797] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:39.378435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d902cb-0c0d-4af8-9c62-cb191f838748 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.424745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.451779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1c57-b370-4797-9d35-c55e476f08d7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.489307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.490125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.490419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.490689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.498219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.499738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eb23ca2d-fa2c-41ea-85f9-4d496e3db5bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:39.515321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:39.516262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:39.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 393/798] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:39.518320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.518571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.518818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.520973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d902cb-0c0d-4af8-9c62-cb191f838748 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.536105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:39.537317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:39.542022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.562233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.563563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.564051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.564524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.573733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.575667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:39.591779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:39.592788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:29:39.593708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:39.600974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 406/799] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:39.602123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.610201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 394/800] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:39.619229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.696927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.710400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f7e12-c456-4ef6-8323-d9c3d935e4ac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.731633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b94d6-15e5-4f52-887b-bb643bd40b66 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.746829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.748289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.749007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.749243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.750146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:39.750554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.752226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.765422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 407/801] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:39.766390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.779994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.780835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.781120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.781397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.782251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:39.782564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.784132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.801602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 395/802] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:39.802379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.808477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.880059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495fa5-deff-434f-82af-3ae20623107c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.910343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b565d5-cba2-4a60-ba05-cc540ceac5bc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:39.914613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.915468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.915753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.916030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.926647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a4e4b22e-a264-4b96-be1e-2189955460bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:39.934310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:39.935586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:39.936530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:39.940755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 408/803] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:39.941530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:39.941861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf3c9-801e-4e5a-b034-07f07a6a9d25 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.942156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf3c9-801e-4e5a-b034-07f07a6a9d25 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.942405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbf3c9-801e-4e5a-b034-07f07a6a9d25 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.947125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 409/804] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:39.947125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbf3c9-801e-4e5a-b034-07f07a6a9d25 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:39.955468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:39.956474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:39.956759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:39.957024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/faa76f7a5fa04d9ca1d2489cc12e9e47/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:39.971905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=faa76f7a5fa04d9ca1d2489cc12e9e47, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:39.973362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:39.999105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 396/805] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:39 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/faa76f7a5fa04d9ca1d2489cc12e9e47/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:39.999105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.058088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.064222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbf3c9-801e-4e5a-b034-07f07a6a9d25 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.110431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.112069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.112974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.113266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.146622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9fc7-8ee8-40f7-a214-fffab121e461 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.188130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.188810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.189096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.189377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.190205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:40.190520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.192892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a082cba-7864-41d4-9b3c-a2fcd0acb4c8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.202364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 397/806] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:40.204013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.204312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.204584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.208138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.209946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.225086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:40.226788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:40.267503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.312083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.322702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 410/807] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:29:40.323064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 398/808] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:40.324263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.333022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.448936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e01b9e-a289-4c04-bb71-67bfd6ec23dc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.472649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba327b5e-020c-46eb-acd0-a3a6c44d3d97 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.489223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.490045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.490330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.490606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.502236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.503883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.511756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.512594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.512852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.513130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:40.522859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:40.523822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:40.524708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:40.524708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:40.524708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:40.524708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:40.524708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 411/809] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:40.525414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.536682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.539033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:40.567096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:40.568013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:40.568906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:40.568906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:40.568906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:40.568906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:40.570533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 399/810] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:40.572170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.582223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.597757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.635208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138242a-b63a-478f-9466-6be5a3d8efe9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.670953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.671875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.672161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.672437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] PATH_INFO: `/v3/services/network` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.673095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.674572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.682213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465c2ee-c788-4d5f-9bea-14585f78818f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-07194935-9764-42e9-b457-683a5ebaf184 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:40.686667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:40.687829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 09:29:40.688655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:40.693136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 412/811] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:40.694257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681d24c-20ae-42ba-930d-b347cee13e9e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.694550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681d24c-20ae-42ba-930d-b347cee13e9e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.694828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681d24c-20ae-42ba-930d-b347cee13e9e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.699122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 413/812] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:40.699122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681d24c-20ae-42ba-930d-b347cee13e9e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.722062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.722739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.723019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.723332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.724040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:40.724345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.725714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.737189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 400/813] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:40.742277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.812066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681d24c-20ae-42ba-930d-b347cee13e9e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.826685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.828024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:40.849028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebedf3-a030-4565-94f8-d849502603f3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:40.850454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.851157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.851445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.851719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.852433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:40.852744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.854130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d49d-f8d5-4e13-8be1-97393bf6a0ea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.867047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 414/814] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:40.868278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.868562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.868838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.885226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:40.886426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:40.893112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:40.894493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:40.894949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:40.895417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:40.896720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:40.897589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:40.899670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:40.905690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 401/815] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:40.907143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:40.954899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 415/816] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:40.956665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.035266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dbd28-fd2b-47b2-b4f0-8cacd2333fac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.063422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bcd20-888f-4440-b946-037c48e2d9e4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.066841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.072732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.073965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.074234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.074523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.084278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.085671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:41.100025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:41.100967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:41.101897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 402/817] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:41.102476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.104978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.105836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.106132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.106386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.107436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.114464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.115981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:41.130634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:41.131583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:41.132465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 416/818] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:41.132942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.217765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec811f-bdde-4bad-bd28-b215aabd08b3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.244202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee83b2-0392-4aa8-ab5c-b889c4932a27 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.253574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.254261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.254540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.254847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.255563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:41.255870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.257250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.266555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 403/819] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:41.266555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.286934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.288220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.288507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.288932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.289595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.290984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:41.303511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:41.304467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 09:29:41.305295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:41.309572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 417/820] 10.222.0.65 () {56 vars in 1129 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:41.310514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.336603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.341339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.372962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e14-f510-4738-bf32-e883ffca29b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.410494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.411267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.411717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.412002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.412854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:41.413166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.414615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.424763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996845ef-3e71-4cf4-b45e-feb4f195d6fb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.433368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 404/821] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:41.434353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.465690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.466635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.466918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.467237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.468176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:41.468781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.470478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.489702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 418/822] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:41.490581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.542313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9f497-dc6f-4260-bc87-e9375c3c2a35 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.570102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.580942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.581858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.582148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.582424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.583202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.584332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.585371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.597127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd925be-ff4e-476b-9d37-2742cdb4f647 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.612940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 405/823] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:29:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:41.614110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.662100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.662914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.663293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.663639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.664433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:41.665391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.666962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.679406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 419/824] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:41.680537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.754510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e243d-2878-4312-9d76-64bb03212034 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.802939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc144d-e200-4bcf-8c05-51fc2ad6162a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:41.803306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.804962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.805388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.806119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.818509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.820557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:41.840496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:41.841444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:41.842350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:41.842350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:41.842350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:41.842350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:41.842350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 406/825] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:41.842620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.856460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:41.858288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:41.858288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:41.858288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:41.859105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:41.859454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:41.861545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:41.868120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 420/826] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:41.868948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:41.887513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.899464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:41.978729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d97-eb51-42fd-9ef9-943fab4d4180 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.016329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78379d18-34f9-4ab3-9100-29d1a9af2251 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.023386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.024078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.024363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.024640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.025366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:42.025677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.027103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.033833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 407/827] 10.222.0.65 () {56 vars in 1124 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:42.038655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.063969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.064970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.065328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.065616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.066566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:42.067265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.069196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.088357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 421/828] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:42.089580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.161121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:42.165398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0379453-a56d-4a24-8d76-88c7c49fbb56 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.167760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:42.200210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c846-9712-4360-b6e9-11e20eb88dd8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.206538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.207519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.207804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.208081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.208832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.210218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503946f-49b1-4498-bf2c-0a085b25b852 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.229531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 408/829] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:29:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:42.230976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0758-2c5e-4efb-9cbd-e11d91dc3438 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.231331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0758-2c5e-4efb-9cbd-e11d91dc3438 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.231612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0758-2c5e-4efb-9cbd-e11d91dc3438 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.232983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 409/830] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:42.238850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.239182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.239468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.255265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.256416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.257101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.257552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.257774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:42.259005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:42.266572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.268690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:42.285748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:42.286742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:42.287657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:42.289997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 422/831] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:42.295385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.349626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 410/832] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:42.377476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.427525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f68bd8-efda-4ecb-befc-605f062e8337 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.465752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:42.475679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.476476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.476813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.477155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.478363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:42.478363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.479817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.481689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:42.494120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 423/833] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:42.500040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.505471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07e75a-5d44-4933-8c1a-cad34c77b398 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.551991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.552958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.553247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.553525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.567537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:42.577276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:42.578449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:42.579521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 411/834] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:42.587186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.614101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7600b9-2953-4032-8117-864d3c93256a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.656562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.657479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.657763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.658043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/616b28448ac548c0b9c22f1245b090b6/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.660211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=616b28448ac548c0b9c22f1245b090b6, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.661697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.683330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 424/835] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:42 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/616b28448ac548c0b9c22f1245b090b6/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:29:42.714281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6aa77-94ce-44f8-967d-25225e270b96 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.754655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.755580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.755962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.756329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.757218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:42.757624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.759162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.769695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 412/836] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:42.775141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.832226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:42.886142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3b4f-e25f-4222-b71e-51685674bd33 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:42.922493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:42.923333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:42.923621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:42.923964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:42.935102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:42.936533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:42.953651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:42.954748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:42.955913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:42.955913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:42.955913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:42.955913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:42.955913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 425/837] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:42.961408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:42.971235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:43.069056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd537a1-debb-4baf-885f-a9f6344f8265 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:43.106725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:43.107442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:43.107729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:43.108008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:43.108741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:43.109048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:43.110447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:43.123658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 413/838] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:43.129221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:43.228289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:43.231752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a280c-e37e-480d-8471-ca56eff0d609 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:43.265701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:43.266542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:43.266827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:43.267186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:43.275647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:43.277198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:43.292287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:43.293212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:43.294556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 426/839] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:43.299628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:43.367321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:43.403698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e944-bb6d-419e-b9c1-0dbeaa0a9f0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:43.438380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:43.439254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:43.439678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:43.440101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:43.441404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:43.441835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:43.443424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:43.463000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 414/840] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:43.469060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:43.588565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9c510-a4ee-4075-85fb-39de2d0d385c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:43.622962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:43.623652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:43.623935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:43.624208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:43.654744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:43.692338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:43.693823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:43.797033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 427/841] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:29:43.802725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:43.908570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4309-17e4-4840-a57f-ca1ce086c954 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:43.936240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:43.948009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:43.948818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:43.949103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:43.949377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:43.959929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:43.961395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-71feed9a-6757-4964-9985-63a2e9a1cbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:43.983097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:43.984015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:43.984882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:43.984882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:43.984882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:43.984882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:43.984882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 415/842] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:43.991350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.030105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33108-4692-4a2f-bd8e-93c9d6428537 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.030430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33108-4692-4a2f-bd8e-93c9d6428537 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.030671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33108-4692-4a2f-bd8e-93c9d6428537 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.032357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 416/843] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:44.038317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.038599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.038872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.055914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:44.057095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:44.063461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.113567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced597d3-3c29-47b3-a91a-081b0f013da5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.139303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 417/844] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:44.150733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.151458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.151744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.152022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.152753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:44.153062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.154471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.154816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.167401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 428/845] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:44.172690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.272445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495a028-9b8d-4d41-b214-953f9628ac2c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.282641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.293582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dde565-ef15-4194-ad54-b8fa939ba2fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.316835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.320563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.322663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.323264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.323686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.325110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.327190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.333276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.334086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.334363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.334629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.341686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:44.342967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:44.344106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:44.345151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:44.346078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.348156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 418/846] 10.222.0.65 () {56 vars in 1132 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:44.354545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:44.367686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:44.368634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:44.369524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 429/847] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:44.375530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.479608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f0478-42d7-4523-ad96-99d0c937ec71 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.485819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b496f1-238f-4d67-bbff-b567f5b77919 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.514406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.515105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.515420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.515710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.516417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:44.516725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.527112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.527112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 419/848] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:44.529070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.537900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.538582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.538863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.539167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.539962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:44.540270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.541741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371504e4-48f2-47b1-8191-f998f959516b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.553935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 430/849] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:44.555063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e042b0-260d-4118-aca0-9e7b4f5e95b2 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.555381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e042b0-260d-4118-aca0-9e7b4f5e95b2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.555660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e042b0-260d-4118-aca0-9e7b4f5e95b2 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.557035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 431/850] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:44.561600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e042b0-260d-4118-aca0-9e7b4f5e95b2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.570605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.646945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875df1c-5476-4601-9ce9-b7b710e5f347 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.684745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.685425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.685713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.685982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.686698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:44.687005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.688448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43414f36-685b-4880-8f41-7a9f13183144 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.699759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 420/851] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:44.699759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.699759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.699759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.720436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:44.722078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:44.751395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.817095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 421/852] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:44.818284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e042b0-260d-4118-aca0-9e7b4f5e95b2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:44.835989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:44.858774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:44.859508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.859789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.860065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.860911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=faa76f7a5fa04d9ca1d2489cc12e9e47) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:44.861319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:44.862952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74db7f9-823b-41af-b232-a35d91fe9327 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:44.870630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 432/853] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/role_assignments?user.id=faa76f7a5fa04d9ca1d2489cc12e9e47&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:44.881304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:44.882965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe948b-1756-461e-a96f-b3579099b671 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.883287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe948b-1756-461e-a96f-b3579099b671 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.883566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe948b-1756-461e-a96f-b3579099b671 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.885087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 433/854] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:44.892882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:44.893229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:44.893704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:44.917835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:44.921022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:44.979605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80df0ad-82e7-4426-8384-57f7d6e91b93 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.009525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 434/855] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:45.028108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.028932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.029214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.029485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.030150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.031577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.036981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:45.044337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:45.045330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:45.046149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 422/856] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:45.053663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.077848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:45.164492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90399b62-d302-4fea-8751-dbaa1614b195 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.167932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:45.177664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663597b-a5ec-4a0f-8b74-b35142ff48e1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.201742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.203100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.203486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.203856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.217557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:45.225024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.225981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.226405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.226828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:45.227220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:45.228264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:29:45.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:45.230028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 435/857] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:45.230852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:45.231379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.232985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.235914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.239765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 423/858] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:45.245632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.351157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:45.358091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911df340-f55d-4941-ad96-e581f48a6f2e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.364292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a2e1-9821-44a9-a518-98841bd82478 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.394750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.395557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.395909 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.396250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.397063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:45.397456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.399144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.408497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.409495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.409952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.410389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.411354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:45.411823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.413096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 436/859] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:45.413696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.414153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.420615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 424/860] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:45.536177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49146-d184-4b79-a0fd-7fec75ab732b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.574885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.575738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.576026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.576306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.587928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.589407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:45.607511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:45.608370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:45.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:45.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:45.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:45.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:45.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 437/861] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:45.615431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.639716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:45.723565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5f83e-12c3-47cc-ac76-5439cfb97e46 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.769885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.770561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.770838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.771148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.771881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:45.772194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.773581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.786942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 425/862] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:45.792130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:45.804934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:45.898805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0088ff-d9d0-497b-89fd-f1460e86e259 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:45.935708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:45.936524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:45.936810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:45.937081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:45.945209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:45.946681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:45.960790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:45.961748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:45.963611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 438/863] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:45.968182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:46.051446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:46.070499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346620cc-06b8-485d-b093-e4027296a201 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:46.104678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:46.105360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.105647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.105926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.106781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:46.107103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:46.108552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:46.127993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 426/864] 10.222.0.65 () {56 vars in 1135 bytes} [Fri Jun 24 09:29:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:46.133896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:46.204117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:46.241187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebdb99-3ac0-4547-9f63-8972a4105590 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:46.275281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:46.275943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.276225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.276500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.345282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:46.346791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:46.412096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:46.449075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 439/865] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:29:46.454771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:46.583989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992daf-b133-41d1-922b-ee6fbe39706d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:46.626197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:46.628049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.628534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.629001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.636211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:46.645176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:46.646664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:46.665444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:46.666410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:46.667950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:46.667950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:46.667950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:46.667950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:46.667950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 427/866] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:46.672253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:46.784192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bd691-c175-4635-95bd-06aeb017e863 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:46.823033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:46.823788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.824076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.824354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.825088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:46.825398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:46.826891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3812ab-4a8a-491f-86e3-011bb1259fec None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:46.842683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 440/867] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:46.844773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:46.919271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d1cc-18f5-491b-bb69-fd27984fbd17 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.919667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d1cc-18f5-491b-bb69-fd27984fbd17 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.919953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d1cc-18f5-491b-bb69-fd27984fbd17 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.922939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 441/868] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:46.928124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.928412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.928687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:46.945832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:46.947449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:46.961731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1111-f223-45de-b11a-a9ea6dc470fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:46.971722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:46.998096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:46.998819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:46.999129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:46.999581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.006704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.008389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.021786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 442/869] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:47.029452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:47.030424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:47.031478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 428/870] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:47.031969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.044960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.099164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.144809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d89fd7-5acd-4a0c-94b6-61a4b1a3c8fe None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.153718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d11da6-3422-4b83-a1a7-8f240194ba05 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.181015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.181793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.182074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.182348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.183151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:47.183460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.185160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df307-5faa-48c3-b23e-e05c6752b935 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.189977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.190656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.190945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.191262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.191986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.193867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6354f-3005-40de-910c-df0a84f4e60a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.202374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 443/871] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:47.202374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c426-1365-4322-b012-edea80c34293 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.202374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c426-1365-4322-b012-edea80c34293 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.202374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c426-1365-4322-b012-edea80c34293 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.202374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 444/872] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:47.207023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2c426-1365-4322-b012-edea80c34293 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.219621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 429/873] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:47 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:29:47.222071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.222571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.223192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.236152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.246823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:47.248511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:47.340993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 430/874] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:47 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:47.343245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2c426-1365-4322-b012-edea80c34293 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.370569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.385178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.386246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.387404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.387885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.388940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=616b28448ac548c0b9c22f1245b090b6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:47.389468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.391183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892f431-dc51-468b-891b-81d5a3e57b9e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.398543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 445/875] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:47 2022] GET /identity/v3/role_assignments?user.id=616b28448ac548c0b9c22f1245b090b6&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:47.410491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.502647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b45a-baf0-4ab6-933b-6f87ca2fdbeb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.543667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.544894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.545386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.545853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.558840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f1a5e877-e2f9-4832-b3c2-6e90d3909e7b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:47.567511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:47.568471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:47.569303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:47.572491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 431/876] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:29:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:47.573437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4685-23b4-416e-aa81-89ea70139c5a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.573889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4685-23b4-416e-aa81-89ea70139c5a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.574006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4685-23b4-416e-aa81-89ea70139c5a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.575986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 446/877] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:47.581925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.582074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4685-23b4-416e-aa81-89ea70139c5a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.582252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.582496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.600230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:47.601422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:47.674661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 432/878] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:47 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:47.687255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.698578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.711215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4685-23b4-416e-aa81-89ea70139c5a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.732665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.755204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.755485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.755770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.756052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.757187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:47.757503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.759262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.775187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 447/879] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:29:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:47.776093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:47.828635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df8cb-96f3-4d92-b899-aecb3ce8f7cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.872260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.872941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.873221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.873492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.874221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.876187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.895976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 433/880] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:47 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:29:47.910893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121405f-ae52-407a-a6f5-57321b46f0d5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:47.962990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:47.964163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:47.964449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:47.964727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:47.974587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:47.976945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:47.978468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:47.998851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:48.000013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:48.001329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:48.001329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:48.001329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:48.001329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:48.001329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 448/881] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:48.008706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:48.104632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:48.121479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01332e-5a8b-4fc0-99a0-a0a6ec0f88bb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:48.157546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:48.158756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:48.159289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:48.159821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:48.160761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:48.161261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:48.162836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:48.175602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 434/882] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:48.182599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:48.283517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8aff-a160-47ac-8c70-72ceae078738 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:48.321363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:48.322196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:48.322484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:48.322763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:48.330427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:48.331925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:48.347710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:48.348681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:48.350054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 449/883] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:48.354957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:48.462419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bcdc3-7354-49f2-af25-f74aeaddc051 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:48.498758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:48.499918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:48.500462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:48.500934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:48.502055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:48.502783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:48.504547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:48.525352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 435/884] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:48.533458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:48.585894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:48.632351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fefaf-9db9-438c-9845-79c522034c5c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:48.666958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:48.667660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:48.667947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:48.668224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:48.738829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:48.740342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:48.791422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:48.839752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 450/885] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:29:48.848178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:48.859368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:48.949745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87054fa6-06b5-4eaf-bb85-c6ad47f908bb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:48.983915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:48.985013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:48.985481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:48.985938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:48.996493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:48.998568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:49.016187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:49.017576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:49.018574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:49.018574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:49.018574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:49.018574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:49.020636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 436/886] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:49.026354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.130116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f1d31-93b2-4ea2-be34-bab3e08191af None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.165244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:49.166178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.166606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.167057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.167947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:49.168401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:49.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.182288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 451/887] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:49.187998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.245440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:49.289802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123dcdf-128a-4207-8140-3c3be972e6e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.323301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:49.324171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.324460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.324781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.334888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:49.336405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bfc518a8-de33-421b-bd64-691a7530014f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:49.351006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:49.351904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:49.352769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 437/888] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:49.358152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.379482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:49.463237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdab9bc-dd92-4e37-9634-accc7b35d357 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.480708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500bfd8-d2a4-4627-8192-9bc55e1e3d93 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.481066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500bfd8-d2a4-4627-8192-9bc55e1e3d93 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.481283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500bfd8-d2a4-4627-8192-9bc55e1e3d93 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.482790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 438/889] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:49.488081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.488370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.488649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.502944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:49.503962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.504543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.505028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.506010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:49.506515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:49.508199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.509579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:49.510776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:49.523122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 452/890] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:49.528747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.603350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 439/891] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:49.614572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:49.622398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.652054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7cf1-b592-4680-bce6-351b3f00e2b8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.686856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:49.692797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:49.694106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.694731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.695495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.696797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] RBAC: Adding query filter params (role.id=33085f00b4eb426ea862c332e899e26e, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=faa76f7a5fa04d9ca1d2489cc12e9e47) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:49.697332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:49.699000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8aa7dc-8c7a-48bd-aac4-15abbf874e76 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.705929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 453/892] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/role_assignments?user.id=faa76f7a5fa04d9ca1d2489cc12e9e47&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=33085f00b4eb426ea862c332e899e26e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:49.726496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df8e9-aff8-4d67-868b-8e4f670c0aac None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.726761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df8e9-aff8-4d67-868b-8e4f670c0aac None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.727060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3df8e9-aff8-4d67-868b-8e4f670c0aac None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.732908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 454/893] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:29:49.734725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.734990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.735325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.756712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:49.757935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:49.795011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff65e4-1552-46c7-a5b1-4be4ddf61a47 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.843398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 455/894] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:49.847148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:49.848331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:49.848820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:49.849295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:49.850203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:49.851890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:49.868471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:49.869617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:49.870558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:49.871428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.879139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 440/895] 10.222.0.65 () {56 vars in 1132 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:49.881713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:49.941641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:49.979693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b6d8e-ce36-4e85-a469-601fa9d04e0f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:49.999338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ebed2-9a59-47f4-b729-91e313b20aff None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.010673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.023812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.024971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.026037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.026511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.027639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.030087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.040333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.041089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.041462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.041807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.042596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.042971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.044451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-1f095f72-023a-4679-8c56-d7ce8f30e98a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:50.048664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:50.049757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:50.050583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:50.051391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 441/896] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:50.052551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 456/897] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:50.056879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.058970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2985b7-2e4e-49b7-9da5-b25ed24a3220 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.059292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2985b7-2e4e-49b7-9da5-b25ed24a3220 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.059756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2985b7-2e4e-49b7-9da5-b25ed24a3220 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.061698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 457/898] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:50.061698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2985b7-2e4e-49b7-9da5-b25ed24a3220 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.172024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab305-0576-4416-8ae5-f6b1340e2d1b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.175924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2985b7-2e4e-49b7-9da5-b25ed24a3220 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.204694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.211524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.212050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.212261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.212483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.212757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.212868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.213046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.213321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.213489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.213769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.214074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.214385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.215177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.215808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89546071-f313-4a80-8d81-37fc9683fbcb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.225108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 458/900] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:50.225279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 442/900] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:50.225368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.225368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.225368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.228127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.241959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:50.243069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:50.271748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.315699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 459/901] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:50.316955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.346188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13176ca7-af9d-4fc9-81af-3413c728e83d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.384130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.385074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.385518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.386080 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.386884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] RBAC: Adding query filter params (interface=public, service_id=6bd998d61b2c4cf39bcdd402c7215f6f, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.387348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.388738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.399063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 443/902] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/endpoints?service_id=6bd998d61b2c4cf39bcdd402c7215f6f&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:50.399063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.428082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54852cf-1395-4173-82f2-19580cd09c31 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.458746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.476278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.485727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.486089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.486287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.487043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.487361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.488944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.494948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 460/903] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:50.495828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.564417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0471741-9d48-4daa-b31f-c08d87c733fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.569381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.627521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402a577-49b0-4c53-ae85-62672a55df4c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.635045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.637147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.637736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.638307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.639238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.641099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:50.660123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:50.661195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:50.662018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:50.665715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 444/904] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:50.671663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.683225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.683927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.684205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.684477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.685176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.686573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.695917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 461/905] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services => generated 1268 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:29:50.807479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98ef4-47d9-4084-a849-919cd689b682 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:50.846958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:50.847799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:50.848086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:50.848363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:50.849170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:50.849484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:50.850908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:50.857001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 445/906] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:50.862741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:50.925051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:50.977024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075aa9d-fd15-4c2e-bde5-7c5fada4a269 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:51.016306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:51.017915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:51.018613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:51.019443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:51.020588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:51.021060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:51.022640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:51.029689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 462/907] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:51.035492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:51.068535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:51.140093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93877-bbbc-4a1a-b212-836c48727d99 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:51.173871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:51.174543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:51.174835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:51.175130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:51.175931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] RBAC: Adding query filter params (interface=public, service_id=e47133f17a3e4b82a5078de47a2b671f, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:51.176249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:51.177634 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e94ebc-9ea0-4205-98ab-bd4ea73b1948 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:51.187189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 446/908] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:29:51 2022] GET /identity/v3/endpoints?service_id=e47133f17a3e4b82a5078de47a2b671f&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:51.192170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:51.294949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fc6f-3fe1-44e9-9572-ea16d584f32f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:51.313185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:51.330333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:51.331018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:51.331331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:51.331605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:51.332293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:51.333691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a8f99-22dd-4274-8067-43e9dd63d61f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:51.340110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 463/909] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:29:51 2022] GET /identity/v3/services => generated 1268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:29:51.858604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749812-9d5b-4922-882a-8a794452c195 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:51.859001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749812-9d5b-4922-882a-8a794452c195 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:51.859217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749812-9d5b-4922-882a-8a794452c195 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:51.860952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 447/910] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:51.866269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:51.866719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:51.866815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:51.885176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:51.886408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:51.955504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 464/911] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:51.978148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749812-9d5b-4922-882a-8a794452c195 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.079578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:52.087359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749812-9d5b-4922-882a-8a794452c195 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.122947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.123793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.124076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.124344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.135213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:52.143923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:52.144872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:52.145710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 448/912] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:29:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:52.153206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.252696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1e053-19c9-489b-9827-3de8ec8cc755 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.263846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:52.287429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.288401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.288684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.288959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.289776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:52.290084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:52.291564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:52.300386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 465/913] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:29:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:52.305900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.415991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cade3ce-a733-4851-aed7-ce70f55992f6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.452664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.454032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.454533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.455869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.468754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:52.471287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:52.490040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:52.491009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:52.491909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:52.491909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:52.491909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:52.491909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:52.494206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 449/914] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:52.500476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.608712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa83f5d-defc-40b3-91c5-05ba1db6d84f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.643509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.644251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.644531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.644813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.645530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:52.645840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:52.647259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ab1a8-4dba-4a3a-a86e-ca745415a8eb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:52.659565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 466/915] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:52.665144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.679512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:52.772907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2fd2d-9954-458e-9155-072a9813c705 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.781915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e964123-4772-4a7a-a107-ea0855f8489b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.782532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e964123-4772-4a7a-a107-ea0855f8489b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.783100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e964123-4772-4a7a-a107-ea0855f8489b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.785023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 467/916] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:52.790255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998819a-070b-494c-a60a-57d28a573c17 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.790531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998819a-070b-494c-a60a-57d28a573c17 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.790804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998819a-070b-494c-a60a-57d28a573c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.808581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8998819a-070b-494c-a60a-57d28a573c17 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:52.808759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.809582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.809769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8998819a-070b-494c-a60a-57d28a573c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:52.809878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.810121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.818843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:52.820348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:52.836030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:52.837014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:52.837969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 450/917] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:52.838563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:52.842946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.887794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 468/918] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:52.890417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc83a8d-bb16-4bc3-b6ea-823e0fbf9c1c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.890801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc83a8d-bb16-4bc3-b6ea-823e0fbf9c1c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.891191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc83a8d-bb16-4bc3-b6ea-823e0fbf9c1c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.893701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 469/919] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:52.898291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.898577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.898851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.916284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:52.917488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:52.918040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:52.956509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf80d3-c3dd-4dfe-941a-f4a01e1a1d4d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:52.991866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:52.992224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 470/920] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:52.992547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:52.992830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:52.993113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:52.993931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:52.994033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:52.994240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:52.995726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.016771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 451/921] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:53.018445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.093897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.104769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4015726-251d-4a58-9d62-b03b546091d7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.122536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b770b1-1f1a-43da-91dd-cd6168089908 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.146905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.147484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.148386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.148780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.149069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.149829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.151862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.159137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.160213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.160498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.160784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.161717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.163127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.178072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 471/922] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:52 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:53.178072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:53.178360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:53.179334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:53.180171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 452/923] 10.222.0.65 () {56 vars in 1132 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:53.183133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.317406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04869639-40e1-4579-882c-8f839f03b59a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.323941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d00ec-1930-45f4-b56e-cbbe1e1aa3ad None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.362927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.363703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.364596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.364596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.364961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:53.365257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.366658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.371959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.372776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.373055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.373334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.375895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 453/924] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:53.382290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.385287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.387636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.397396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a2a5675c-1e08-4195-b41c-aa5e0cbcd86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:53.407538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:53.409177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:53.410046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:53.410046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:53.410046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:53.410046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:53.410046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 472/925] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:53.411213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28632869-c4b5-422c-98dc-f7d38fa8af6f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.411436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28632869-c4b5-422c-98dc-f7d38fa8af6f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.411723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28632869-c4b5-422c-98dc-f7d38fa8af6f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.413371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 473/926] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:53.417055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28632869-c4b5-422c-98dc-f7d38fa8af6f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.457339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.510845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5342255-780b-48ee-9dc4-f448764b173d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.528701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28632869-c4b5-422c-98dc-f7d38fa8af6f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.551065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.551774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.552061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.552339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.553051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:53.553371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.554757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935310c9-7a68-42b9-82f5-d1d67e24d1fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.565247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 454/927] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:53.565247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.565247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.565247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.572916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.573607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.573899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.574591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.575457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:53.575727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.577039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.579562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:53.580805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:53.593826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 474/928] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:53.593826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.655593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.666624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 455/929] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:53.667836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.708262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c977b-4c33-481f-81d6-3adb21def188 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.725635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.751181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.752200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.752688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.752968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.754583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.756227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.782962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 475/930] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:29:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:53.784078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.785097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f062a8-e16b-4cef-aace-8699fefee68b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.833448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.834381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.834807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.835211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.843692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.845515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:53.861342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:53.862501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:53.863648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:53.865618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 456/931] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:53.870947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.904029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a19c66-b226-484a-9d4f-7cb1d65f869f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.922860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:53.946442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:53.947363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:53.947731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:53.948008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:53.948677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:53.950080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:53.963921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:53.964951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:53.965781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 476/932] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:53.971169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:53.994977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400073ba-880c-4837-841c-dca9b1874b09 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:53.997111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:54.051793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:54.052841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:54.053353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:54.053826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:54.054852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:54.055385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:54.057202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:54.072154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 457/933] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:54.077924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:54.097176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480adf-8823-4c91-be88-306b7d518f03 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:54.133097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:54.134118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:54.134645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:54.135149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:54.136106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:54.136653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:54.138260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:54.143766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 477/934] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:54.149010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:54.200459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3237-d098-45da-ad82-2a383e4664ee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:54.212759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:54.238274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:54.239594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:54.240099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:54.240574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/faa76f7a5fa04d9ca1d2489cc12e9e47/roles/33085f00b4eb426ea862c332e899e26e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:54.243237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=faa76f7a5fa04d9ca1d2489cc12e9e47, role_id=33085f00b4eb426ea862c332e899e26e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:54.244871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:54.262979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 458/935] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:29:54 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/faa76f7a5fa04d9ca1d2489cc12e9e47/roles/33085f00b4eb426ea862c332e899e26e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:29:54.281863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489824d1-b4d2-448b-a91a-302e11e3a078 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:54.321981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:54.322699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:54.322982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:54.323293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:54.324035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:54.324342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:54.325817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427d114-b3e5-449d-b0e7-4b1503906c9b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:54.332479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 478/936] 10.222.0.65 () {56 vars in 1114 bytes} [Fri Jun 24 09:29:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:54.339337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:54.368752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:54.455808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa124b1-db14-48ca-bf07-70c46f85f8fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:54.493718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:54.494392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:54.494675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:54.494949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:54.495736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:54.497563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a410c-4a37-4435-9d35-329ca8c8cc70 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:54.527505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 459/937] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:29:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:55.340773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4eee-7689-445a-84e6-f2fb82e41874 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.341839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4eee-7689-445a-84e6-f2fb82e41874 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.342323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4eee-7689-445a-84e6-f2fb82e41874 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.343942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 479/938] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:55.349794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.350622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.351137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.369109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:55.370293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:55.447655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 460/939] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:55.470958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c4eee-7689-445a-84e6-f2fb82e41874 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:55.577680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c4eee-7689-445a-84e6-f2fb82e41874 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:55.587485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:55.613290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:55.614435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.614916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.615433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.616289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:55.617864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:55.630797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:55.631862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:55.632725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:55.635574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 480/940] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:55.641782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:55.647417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:55.752283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5fa3b-4f69-4cc4-b0dc-0f023947b31a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:55.787329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:55.788003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.788288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.788562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.789270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:55.789578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:55.790974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b688cf0-3e38-4ce5-96df-5b543cbe13d5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:55.796259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 461/941] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:55.801860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:55.892209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514063b4-8f48-43c3-b54e-3a6bd85f4912 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.892834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514063b4-8f48-43c3-b54e-3a6bd85f4912 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.893721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514063b4-8f48-43c3-b54e-3a6bd85f4912 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.896067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 462/942] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:55.901406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.901934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.902427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.914414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eae98-0ccb-4a0d-aa7c-ac873d436823 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:55.919963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:55.921376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:55.948161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:55.948843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:55.949125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:55.949403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:55.950118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:55.950422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:55.951844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:55.961827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 481/943] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:55.963452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.001853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.006309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.020648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 463/944] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:56.036997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.080922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9958b-443f-484b-8b36-ee8f79ab3a55 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.127936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.128982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.129465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.129934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.130982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] RBAC: Adding query filter params (interface=public, service_id=6345c04e0b4649a0bd3dc729a9bf29c4, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:56.131976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.134618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.143120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 482/945] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:29:55 2022] GET /identity/v3/endpoints?service_id=6345c04e0b4649a0bd3dc729a9bf29c4&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:56.148828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.167003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4cc5c-d5d9-4bc9-9289-26800f3858c3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.206968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.208467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.209112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.209898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.210780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.212832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:56.228433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:56.229465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:29:56.230301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:56.237706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 464/946] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:56.239522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.264415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.269028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3164-ad0f-4836-8f04-4843c91691bd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.303831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.309631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.310584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.311095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.311609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.313339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.315133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.322370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 483/947] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:29:56.355950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954dee5-8536-46cf-af41-32d1d8cae7e3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.395561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.396616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.397129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.397621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.398565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:56.399142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.400764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c7f0-5c90-4844-adde-2a327b66cddf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.406975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 465/948] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:56.413417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.435464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f5cb-301b-4b4a-ba03-80da1e21b04c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.435788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f5cb-301b-4b4a-ba03-80da1e21b04c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.436039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588f5cb-301b-4b4a-ba03-80da1e21b04c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.437728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 466/949] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:56.444377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.444700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.444954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.467578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:56.469264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:56.541391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da10f0-6522-401e-b563-74820c722b4d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.561387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 467/950] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:56.582355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.583446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.583963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.584559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.586501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.587284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:56.587868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.589573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91893e74-d499-418b-8ab8-8bfa7b450880 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.595110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 484/951] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:56.605275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8682596-83f8-4245-8069-bf127fdb041e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.605617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8682596-83f8-4245-8069-bf127fdb041e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.605856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8682596-83f8-4245-8069-bf127fdb041e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.607629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 485/952] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:56.617425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f670c-d508-4560-ad02-8750d83628da None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.617764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f670c-d508-4560-ad02-8750d83628da None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.617979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f670c-d508-4560-ad02-8750d83628da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.647280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-078f670c-d508-4560-ad02-8750d83628da None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:56.648583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-078f670c-d508-4560-ad02-8750d83628da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:56.662139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.683057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.727218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91473442-fcef-4f4b-86a0-b113c05f378e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.752742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 486/953] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:56.779653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f670c-d508-4560-ad02-8750d83628da None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.792131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.793223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.793223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.793509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.805652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:29:56.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:56.820270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:56.821126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 468/954] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:56.823232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:56.895816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f670c-d508-4560-ad02-8750d83628da None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.937742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:56.938466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:56.938799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:56.939086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:56.939852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:56.941314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:56.942843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING keystone.server.flask.common [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 09:29:56.962870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.sql.core [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:29:56.962870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 09:29:56.962870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [parameters: {'id': '7ce3af721bc84623b998f78f61b1a1f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 09:29:56.962870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80359) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:29:56.968019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba15cd-44c8-4fea-9360-8cfd934f67e5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ce3af721bc84623b998f78f61b1a1f8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ce3af721bc84623b998f78f61b1a1f8. Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:56.986562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 09:29:56.987488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application yield resource Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise self.value Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application session.commit() Jun 24 09:29:56.988434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:29:56.990124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:29:56.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:56.992380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [parameters: {'id': '7ce3af721bc84623b998f78f61b1a1f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.993323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:56.994294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ce3af721bc84623b998f78f61b1a1f8. Jun 24 09:29:56.995380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:56.995884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:56.995956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 487/955] 10.222.0.65 () {60 vars in 1140 bytes} [Fri Jun 24 09:29:56 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:56.995956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.014149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:57.020403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.021470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.021947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.022448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.023503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:57.024185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.027118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.038731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 469/956] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:57.044338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.125199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2663ab5-e932-45d6-9452-2c53a131460e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.171517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362fa24-468e-45d4-a045-5013ec2bbde1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.174631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.175725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.176212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.176783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.186370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.188139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:57.204581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:57.205720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:57.207141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:57.210436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 488/957] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:57.211292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.217670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.218779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.219332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.219846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.236951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.239257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:57.263230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:57.264345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:57.265530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:57.265530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:57.265530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:57.265530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:57.267928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 470/958] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:57.275112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.296191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:57.322928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cff8d-ae38-41e6-a39d-def5ef7e737b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.323177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:57.361369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.362444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.363006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.363627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.364588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:57.365118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.366886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.395492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 489/959] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:57.409412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78ad44-2899-4d83-8f96-dbfc11205fdb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.449379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.449996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.450274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.450551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.451333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:57.451708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.453210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.467929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 471/960] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:29:57.474067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.592011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b79974-731c-4789-8d1c-ce8d5e19dc5b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.627588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.628430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.628710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.628985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.631289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:57.637411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.638865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:29:57.654551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:57.655661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:57.656596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 490/961] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:57.661867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.764690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:57.775952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a7d6b-3e3d-4b54-81b1-91172a33d2ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.810934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.811669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.811960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.812244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:57.813098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:57.813416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:57.815135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:57.833190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 472/962] 10.222.0.65 () {56 vars in 1149 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:57.838575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:57.943130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974efde-0da9-49e2-a66a-ad8ac12e83ea None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:57.979973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:57.980647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:57.980937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:57.981214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.045132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:58.057712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.059220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.160766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 491/963] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:29:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:29:58.166265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.270931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14920dc-6c6f-4078-9374-968f1eb18ef9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:58.306790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:58.307554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.307838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.308117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.318839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.320347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.329204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-15415db9-f324-4cd7-8e0b-fac4bbc8f64f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:29:58.339699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:58.340590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:58.341496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:29:58.341496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:29:58.341496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:29:58.341496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:58.341496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 473/964] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:58.347223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.449588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:58.456993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc4ecd-ea42-4c3b-a6a3-06507ebb7818 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:58.484820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00364-24da-4264-91ac-f5b100ba6b77 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.486861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00364-24da-4264-91ac-f5b100ba6b77 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.487421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00364-24da-4264-91ac-f5b100ba6b77 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.489388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 474/965] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:58.494779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.495353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.495852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.503336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:58.504018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.504298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.504575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.505293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:58.505597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.506975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.515645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:58.517327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:58.519471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 492/966] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:58.525549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.609980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 475/967] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:29:58.635878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.654588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73f65-d242-46dc-a536-afb11dc07ae9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:58.692516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:58.693619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.694049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.694472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.701927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.703632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-065851d6-3b98-4d9e-b17e-f7692edf3e18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:29:58.718379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:58.719357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:58.720393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 493/968] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:58.722012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f360e-c5cf-4bcc-af61-fe5e898c92e2 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.722012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f360e-c5cf-4bcc-af61-fe5e898c92e2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.722012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f360e-c5cf-4bcc-af61-fe5e898c92e2 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.723097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 494/969] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:58.727563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f360e-c5cf-4bcc-af61-fe5e898c92e2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.743728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:58.756661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8fb5d-0cb2-4a8e-8a67-2894c2ce88ee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:58.792964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:58.800826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:58.801653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.801936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.802210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.802869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.804275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1098f2d8-df5e-4a0e-b6a1-74e771397e91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:58.818315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:29:58.819607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:29:58.820870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:29:58.824435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 476/970] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:29:58.825674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.825926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.826235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.843269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:58.843488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f360e-c5cf-4bcc-af61-fe5e898c92e2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:58.844457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:58.878237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:58.878914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:58.879296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:58.879642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:58.880499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:58.880882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:58.882334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:58.894194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 495/971] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:58.894835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.919135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 477/972] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:58.919988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:58.987313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:59.016003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a8047-9687-422b-9401-87b3405543a0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.023141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f556d3-dcdd-4f21-9eb0-11853dea0045 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.047978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:59.051358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.052238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.052679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.052960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.053679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:59.053991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.055413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.064930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.065106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 496/973] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:59.065194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.065279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:59.065383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.065383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.065383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] RBAC: Adding query filter params (role.id=33085f00b4eb426ea862c332e899e26e, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=faa76f7a5fa04d9ca1d2489cc12e9e47) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:59.065383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.065383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.070521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 478/974] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:29:58 2022] GET /identity/v3/role_assignments?user.id=faa76f7a5fa04d9ca1d2489cc12e9e47&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=33085f00b4eb426ea862c332e899e26e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:59.071369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:59.187532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa6abe-c28c-4314-91ce-9ddef59d7c25 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.220540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20a0ee-d497-4517-b01d-191117e59e25 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.226397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.227210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.227568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.227918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.228708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:29:59.229103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.230574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.236926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 479/975] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:29:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:29:59.242943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:59.270884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.271819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.272202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.272576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.273430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.274922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343cde7-5e62-4c17-9f86-22b344b8a317 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.287117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:59.295787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 497/976] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:29:59 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:29:59.318330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:29:59.381453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac61462-0c63-4e47-a374-d7af495d8aa1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.428282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.429402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.429920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.430426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.431455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.433073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680900b6-e798-41c1-9c45-0cad5ff50e68 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.460486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 480/977] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:29:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:29:59.617298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb586-9ab1-4a84-b66e-b647f53b8d64 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.617852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb586-9ab1-4a84-b66e-b647f53b8d64 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.617852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb586-9ab1-4a84-b66e-b647f53b8d64 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.619496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 498/978] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:29:59.625450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.625881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.626239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.644603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:29:59.645810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:29:59.726534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 481/979] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:29:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:29:59.763563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb586-9ab1-4a84-b66e-b647f53b8d64 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:59.879037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb586-9ab1-4a84-b66e-b647f53b8d64 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:29:59.915372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:29:59.916063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:29:59.916348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:29:59.916622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:29:59.917341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:29:59.918792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:29:59.933267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.sql.core [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:29:59.933267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 09:29:59.933267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [parameters: {'user_id': 'ec148146c4514522a1fbbdde65f96e4d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 09:29:59.933267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80359) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:59.941563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:59.942590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application yield resource Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise self.value Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application session.commit() Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 09:29:59.943941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.flush() Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application compat.raise_( Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 09:29:59.944850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 09:29:59.945871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 09:29:59.946716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ec148146c4514522a1fbbdde65f96e4d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:29:59.949157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 09:29:59.950962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:29:59.957472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 499/980] 10.222.0.65 () {60 vars in 1131 bytes} [Fri Jun 24 09:29:59 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:29:59.961166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:29:59.994468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:00.075788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bbcd3-02df-478c-8098-60a760f3aaf5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:00.113201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:00.114019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:00.114307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:00.114586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:00.122669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:00.124188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8c8def41-b25b-474c-8b24-4130c7bbf24f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:30:00.139605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:00.141631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:00.144435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 482/981] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:29:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:00.152006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:00.215793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:00.260621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affb9de-3c9f-4196-84d1-a346e4714aea None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:00.298041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:00.298866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:00.299031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:00.299293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:00.300102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:00.300411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:00.302190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13ead6-3a3a-4252-9fc9-6d7c1a7a389b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:00.340470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 500/982] 10.222.0.65 () {56 vars in 1101 bytes} [Fri Jun 24 09:30:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:01.190868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35723-91e5-4ed1-901a-4351d58b1675 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.191652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35723-91e5-4ed1-901a-4351d58b1675 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.191652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35723-91e5-4ed1-901a-4351d58b1675 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.193036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 483/983] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:01.198333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20ebc-bfd1-41dc-860c-ba82fd65cb6c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.198885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20ebc-bfd1-41dc-860c-ba82fd65cb6c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.198885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20ebc-bfd1-41dc-860c-ba82fd65cb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.216434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5ed20ebc-bfd1-41dc-860c-ba82fd65cb6c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:01.217989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5ed20ebc-bfd1-41dc-860c-ba82fd65cb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:01.285331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 501/984] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:01.307355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35723-91e5-4ed1-901a-4351d58b1675 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.402506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:01.414566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35723-91e5-4ed1-901a-4351d58b1675 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:01.450547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:01.451299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.451584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.451864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.452520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:01.453917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:01.459776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10174472-0e76-46cf-a60a-30028f454944 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.460194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10174472-0e76-46cf-a60a-30028f454944 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.460398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10174472-0e76-46cf-a60a-30028f454944 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.462088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 502/985] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:01.467016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:01.468027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:01.469336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:01.471426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.471571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 484/986] 10.222.0.65 () {56 vars in 1105 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:01.471708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.471708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.475899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.486996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:01.488234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:01.547183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:01.575120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 503/987] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:01.596353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.606816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ddcb-4ce6-4785-b0c7-d0ccf435628c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:01.640869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:01.641541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.641888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.642169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.642894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:01.643235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:01.644628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c250f-46b4-4337-a6ee-459bdc6febba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:01.649532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 485/988] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:01.650725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9577-999a-41de-914b-72fc4cb698da None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.651017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9577-999a-41de-914b-72fc4cb698da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.651329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd9577-999a-41de-914b-72fc4cb698da None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.652579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 486/989] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:01.655156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9577-999a-41de-914b-72fc4cb698da None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.673865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:01.704434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdcee5-7520-477e-86a0-74ee6f821f80 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:01.738702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:01.739554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.739828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.740123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.740786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:01.742173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-90e1c669-a9f8-4966-8251-b72ebd5d45a1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:01.755026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:01.756117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:01.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 504/990] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:01.757751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.757830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.758188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.768304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd9577-999a-41de-914b-72fc4cb698da None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:01.777430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:01.778624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:01.803239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:01.803916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.804202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.804485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.805201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:01.805553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:01.806945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:01.811567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 487/991] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:01.812338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.817138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:01.857063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 505/992] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:01.882407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:01.933100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:01.955408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807cb2-3f1c-44ec-a070-a332e2658591 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:01.993801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:01.994546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:01.994767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:01.995299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:01.996013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:01.996693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:01.998076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.003849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 488/993] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:02.005067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d34088-27f4-4fff-814f-91898baa3365 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.010684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.044400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.045209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.045489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.045763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.046413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.047853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:02.059836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:02.060848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:02.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 506/994] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:30:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:02.066878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.102174 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:02.129795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9570-f211-4ab4-b36b-085b94838955 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.165070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.165911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.166512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.166802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.167551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:02.167864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.169356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.172909 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2001f7f-5bf1-4d4d-8e25-5d1a2c3cc442 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.174707 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 489/995] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:02.180224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.192891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:02.209308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.210001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.210241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.210524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.211271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:02.211586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.212986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.218376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 507/996] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:02.223721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.295624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addbd7d-4dae-4ba2-8119-d67734a163b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.327907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:02.330810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d140b91-2023-4df6-8273-f733526c399a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.334056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.334725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.335004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.335314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.336136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] RBAC: Adding query filter params (interface=public, service_id=0985ac29ffdd4050b0566ed2f7bdbff7, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:02.336449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.337877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.345635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 490/997] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/endpoints?service_id=0985ac29ffdd4050b0566ed2f7bdbff7&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:02.351326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.376021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.376709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.377151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.377432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.378269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:02.378651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.380116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.390112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 508/998] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:02.391219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.431051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:02.457187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53185cc-4dc9-475a-b5ea-a8c0283168f0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.495837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.496796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.497078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.497343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.498049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.499461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.500578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7938b84-0fe9-430a-81f2-84401d756b82 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.506102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 491/999] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:30:02.539429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.540125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.540414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.540694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.541514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] RBAC: Adding query filter params (interface=admin, service_id=6345c04e0b4649a0bd3dc729a9bf29c4, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:02.541841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.543355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d73d75-c33b-4d8b-b671-1ab6c60f8457 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.555134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 509/1000] 10.222.0.65 () {56 vars in 1249 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/endpoints?service_id=6345c04e0b4649a0bd3dc729a9bf29c4&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:02.556459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:02.669378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7d900-9081-4e9b-8a48-4c6555c1888e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:02.704872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:02.705873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:02.706343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:02.706805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:02.707709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:02.709276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852ad94-8c38-41c0-9ce1-58d160eda3e6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:02.716187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 492/1001] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:30:02 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:30:03.851681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e103af-646a-46b0-af69-6ea4c8e8d2a6 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:03.852224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e103af-646a-46b0-af69-6ea4c8e8d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:03.852224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e103af-646a-46b0-af69-6ea4c8e8d2a6 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:03.853792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 510/1002] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:03.858955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fbee8-2a1f-44da-9e59-26a8758767d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:03.859405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fbee8-2a1f-44da-9e59-26a8758767d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:03.859573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fbee8-2a1f-44da-9e59-26a8758767d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:03.864862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:03.883518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0e6fbee8-2a1f-44da-9e59-26a8758767d3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:03.887410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0e6fbee8-2a1f-44da-9e59-26a8758767d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:03.957896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 493/1003] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:03.980966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e103af-646a-46b0-af69-6ea4c8e8d2a6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:04.010868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:04.086432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e103af-646a-46b0-af69-6ea4c8e8d2a6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:04.120350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:04.121018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.121643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.121914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.122639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:04.124045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca38ab-350f-495e-8605-699a6cd07687 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:04.141105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 511/1004] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:30:03 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:30:04.586341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7f393-bf36-4440-84de-a6ecf2e249c6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.586593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7f393-bf36-4440-84de-a6ecf2e249c6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.586900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7f393-bf36-4440-84de-a6ecf2e249c6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.588474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 494/1005] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:04.597177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b47ac-85aa-4d4d-b217-905886a6044d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.597478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b47ac-85aa-4d4d-b217-905886a6044d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.597745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b47ac-85aa-4d4d-b217-905886a6044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.617120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-592b47ac-85aa-4d4d-b217-905886a6044d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:04.618303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-592b47ac-85aa-4d4d-b217-905886a6044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:04.708691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 512/1006] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:04 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:04.731202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7f393-bf36-4440-84de-a6ecf2e249c6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:04.749614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409d60-569b-42c7-b7ef-27fe2de6c67d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.749911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409d60-569b-42c7-b7ef-27fe2de6c67d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.750524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1409d60-569b-42c7-b7ef-27fe2de6c67d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.752129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 513/1007] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:30:04.756944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.757338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.757626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.775406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:04.776602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:04.787351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:04.848348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 514/1008] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:04 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:04.852257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7f393-bf36-4440-84de-a6ecf2e249c6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:04.871562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:04.886696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:04.891815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:04.895413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:04.895852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:04.896305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:04.897121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:04.898680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:04.910630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:04.911787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:04.912626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 495/1009] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:30:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:04.917718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:04.986440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9017a6-7f74-4926-88dc-a10f83ddbed3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.024182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.025791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.026223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.026640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.027290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e7456-a219-48b8-974f-ec776a49445d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.027644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.029333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:05.044406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:05.045537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:05.046371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 515/1010] 10.222.0.65 () {56 vars in 1126 bytes} [Fri Jun 24 09:30:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:05.051840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:05.063254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:05.067243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.067932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.068221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.068507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.069215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:05.069526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.070911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.076182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 496/1011] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:30:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:05.081631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:05.143837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:05.157504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cef54-6505-4e9d-abe8-c2ea5bbf3f66 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.187666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342a5ed-93da-4c7c-b0ff-0888c5e5f8db None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.193060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.193742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.194027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.194305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.195449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:05.195762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.197513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.203467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 516/1012] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:30:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:05.208958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:05.225801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.226489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.226771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.227058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.227796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:05.228106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.229484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.234516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 497/1013] 10.222.0.65 () {56 vars in 1112 bytes} [Fri Jun 24 09:30:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:05.240951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:05.286620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:05.316949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2733-df4a-4e0f-b8aa-4fb6c610de26 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.353131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15adc-e92c-4283-bba4-79854aa8d333 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.353461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.354119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.354403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.354680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.355421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:05.355730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.357177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.362417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 517/1014] 10.222.0.65 () {56 vars in 1122 bytes} [Fri Jun 24 09:30:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:05.367830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:05.382061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:05.396264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.397221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.397715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.398169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.399104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.400688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7209-d62a-40de-924c-30b7b1c29e07 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.422355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 498/1015] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:30:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:05.474591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c004f-da00-4f21-affe-487e4cb744e9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:05.513741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:05.520745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:05.521459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:05.521742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:05.522021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:05.522774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:05.524218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b3ef0-6028-4fe5-81eb-5b5f07606102 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:05.541471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 518/1016] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:30:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:06.195210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e078-449e-4ef6-9b1d-2d0412142cfe None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.195759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e078-449e-4ef6-9b1d-2d0412142cfe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.195759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e078-449e-4ef6-9b1d-2d0412142cfe None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.197124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 499/1017] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:06.202647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.203118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.203309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.220717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:06.221928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:06.302047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 519/1018] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:06.324869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748e078-449e-4ef6-9b1d-2d0412142cfe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:06.436696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748e078-449e-4ef6-9b1d-2d0412142cfe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:06.472271 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:06.473004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.473287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.473558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.474205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:06.475629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:06.488056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:06.489061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:06.489909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 500/1019] 10.222.0.65 () {56 vars in 1105 bytes} [Fri Jun 24 09:30:06 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:06.497591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:06.602663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa959b-882d-475b-b01a-062716c6eaf5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:06.619492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:06.639316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:06.640047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.640335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.640609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.641436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] RBAC: Adding query filter params (name=image) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:06.641727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:06.643122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:06.651172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 520/1020] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:06.654862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:06.710164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:06.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044104e3-4b91-4201-80cd-61fd99c4eb05 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:06.823490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:06.824175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.824462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.824740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.827294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:06.827612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:06.832475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:06.839363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 501/1021] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:06.845426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:06.954862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de807676-bc29-431c-9c1d-6e2a0c5cf486 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:06.990212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:06.991300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:06.991801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:06.992307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:06.993432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] RBAC: Adding query filter params (interface=public, service_id=bc5b37a4bc8a49bebac33bfcd23db4eb, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:06.993950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:06.995665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e073-253b-49fa-a1f0-c15ea1f505a9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:07.002370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 521/1022] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:30:06 2022] GET /identity/v3/endpoints?service_id=bc5b37a4bc8a49bebac33bfcd23db4eb&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:07.008481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:07.132314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad447850-f277-4624-ba61-0b61f3fd4543 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:07.142622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:07.176401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:07.177073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.177354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.177624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.178304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:07.179970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f6f9-cdac-425f-9204-090b4bee24d9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:07.186762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 502/1023] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:30:07 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:30:07.639619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76b2f-97b2-4a97-92e5-32d6f45ac51b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.640214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76b2f-97b2-4a97-92e5-32d6f45ac51b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.640214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76b2f-97b2-4a97-92e5-32d6f45ac51b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.641846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 522/1024] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:07.647534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aed4d-9dc1-479a-86d2-be448af9e72e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.647978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aed4d-9dc1-479a-86d2-be448af9e72e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.648083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aed4d-9dc1-479a-86d2-be448af9e72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.670677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-144aed4d-9dc1-479a-86d2-be448af9e72e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:07.671907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-144aed4d-9dc1-479a-86d2-be448af9e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:07.756720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 503/1025] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:07.781691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e76b2f-97b2-4a97-92e5-32d6f45ac51b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:07.831026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:07.832092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c55c-a8c7-4770-a37c-294491512acc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.832573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c55c-a8c7-4770-a37c-294491512acc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.832970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81c55c-a8c7-4770-a37c-294491512acc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.834543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 504/1026] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:30:07.840119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.840402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.840714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.857732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:07.858943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:07.891532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e76b2f-97b2-4a97-92e5-32d6f45ac51b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:07.927561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:07.928389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:07.928667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:07.928922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:07.934899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 505/1027] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:07.939776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:07.947800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:07.948715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:07.955119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 523/1028] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:30:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:07.955946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:07.960673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:07.988098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.060786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0845e-6507-4f1b-9b9a-6a94dd65643d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.066743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e69b3-d334-4e69-a425-9622ed3ee873 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.074998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.098104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.098784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.099069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.099371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.100132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:08.100430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.101830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.105435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.106238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.106522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.107223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.108108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.109538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.110664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 506/1029] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:30:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:08.111728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 524/1030] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:08.117374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.119589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.221386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.228495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa282cd8-f211-4f3d-9099-dfe703a717d3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.231384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6b0cc-ce67-4824-8343-b8d37e437a11 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.264519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.265189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.265473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.265744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.267013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.268120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.270381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.271488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.271775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.272467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.285030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.286748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.300162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 507/1031] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:30:08 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:08.309993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:08.311122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:08.312032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:08.312032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:08.312032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:08.312032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:08.312789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 525/1032] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:08.318702 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.431514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9201f-c2ed-4f22-a010-fa6cd17cfb37 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.473753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.474363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.474646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.474926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.475750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:08.476059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.477444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.489574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 508/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:08.495505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.524300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.596971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c213e-65f6-4fa0-85be-23c47d6f4c2d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.630987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.631823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.632105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.632381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.639774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.641414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:30:08.656283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:08.657293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:08.658251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:08.658806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 526/1034] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:30:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:08.663885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.674613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.773993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697fd9e-6cd5-4c5d-9092-0f66bd50b267 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.809457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.810132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.810410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.810685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:08.811551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:08.811846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:08.813765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:08.832046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 509/1035] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:30:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:08.837711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:08.927234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:08.945671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5c39-f385-46f2-8752-7e6897a6ffa7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:08.980398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:08.981061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:08.981343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:08.981619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.056555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.057937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f199d-d387-4519-960b-d0cb66ce617d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.144890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.166648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 527/1036] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:09.168173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.231722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.235684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b21f-f8ac-44b6-9c28-cb811a1f0f72 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.235982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b21f-f8ac-44b6-9c28-cb811a1f0f72 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.236267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b21f-f8ac-44b6-9c28-cb811a1f0f72 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.237947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 528/1037] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:09.242997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.243433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.243718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.260466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:09.261788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:09.276279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33410e5-7fb5-4b42-b6a7-fb580771aadb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.313405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.314196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.314496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.314769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.325752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.327360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.331147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 529/1038] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:09.344912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:09.345854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:09.346823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:09.346823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:09.346823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:09.346823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:09.350747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 510/1039] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:09.351933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.354155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.455587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b4ea-f83e-463d-9fbd-9472ed788b2d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.458889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e638a-cf9e-40cc-af1c-0a2c4cb5cd33 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.477430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.477860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.498457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.499298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.499584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.499862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.500390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.500506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.501242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.501534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.501817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.502006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.502549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:09.502861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.504946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:09.513686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:09.514684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:09.515588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 511/1040] 10.222.0.65 () {56 vars in 1105 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:09.520140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 530/1041] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:09.521123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.526735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.630154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a573-cc29-452a-8966-dd92ddb62c86 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.635867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfff71-85d6-4fa5-91a9-45e14c2e0a10 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.666203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.666999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.667402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.667690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.668405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:09.668719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.669860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.670242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.670669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.671060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.671272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.680163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 512/1042] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:09.680307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.680307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.681105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:30:09.694158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:09.700016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 531/1043] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:09.700919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.736717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.739222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.806567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b3ab5-7988-4ce2-ada6-0d183f3bb6fa None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.807454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cbf8d-ce88-4072-975c-3875b06f751d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.842780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.843490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.843776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.844057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.844932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:09.845279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.846778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.847067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:09.848001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:09.848438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:09.848860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:09.849719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:09.850172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:09.851761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:09.856882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 513/1044] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:09.864034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 532/1045] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:09.864034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.864778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:09.965307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:09.972466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966cf9-7efa-48b7-bf55-db1eb28f9754 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.973629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44826202-ccd0-49f0-92e9-93641b58467d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:09.983108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:10.007671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:10.008300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.008585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.008914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.009807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:10.011021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:10.011282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57792970-3153-4a0b-9c85-4935f05dfc24 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:10.011716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.012002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.012280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.013114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] RBAC: Adding query filter params (role.id=33085f00b4eb426ea862c332e899e26e, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:10.013427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:10.014893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef856-7b37-4d61-aad6-f36f2aeffa74 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:10.021664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 514/1046] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:09 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=33085f00b4eb426ea862c332e899e26e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:10.034434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 533/1047] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:30:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:10.410012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542065-0aa7-406a-9567-60458c2d98aa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.410335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542065-0aa7-406a-9567-60458c2d98aa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.410618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542065-0aa7-406a-9567-60458c2d98aa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.412112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 515/1048] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:10.417709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.418008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.418281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.436255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:10.437487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:10.518131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 534/1049] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:10.543544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02542065-0aa7-406a-9567-60458c2d98aa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:10.651729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02542065-0aa7-406a-9567-60458c2d98aa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:10.687860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:10.688846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.688929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.689155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.690359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:10.691854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fabd5-aa44-4f04-b621-15f7cb78d9d7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:10.693655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 516/1050] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:10.699348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:10.785478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:10.809203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55efbca-7b5b-40cd-887a-836e2c7714a3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:10.845684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:10.846372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:10.846654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:10.846936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:10.847750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:10.849209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca500b68-43c6-4450-9a0b-f89c96069bbb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:10.879100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 535/1051] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:30:10 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:30:12.048461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5138-9967-426c-97f7-71043fa250cf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.048982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5138-9967-426c-97f7-71043fa250cf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.048982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5138-9967-426c-97f7-71043fa250cf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.050258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 517/1052] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:12.055950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.056409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.056489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.075246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:12.076472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:12.100632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de3188-6fd6-4f61-a7c2-9b3d70a91501 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.100919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de3188-6fd6-4f61-a7c2-9b3d70a91501 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.101199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de3188-6fd6-4f61-a7c2-9b3d70a91501 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.102596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 518/1053] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:12.107477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.107848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.108201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.126814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:12.128145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:12.158372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 536/1054] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:12.165726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.181415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.207132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 519/1055] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:12.226317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.247685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.290599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28530866-944f-45c0-84a1-27b64d381b81 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.326023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.326778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.327062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.327401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.335920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75a01-579f-46b7-bb48-f6b763b594ac None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.337796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:12.346229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:12.347217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:12.348189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 537/1056] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:12.353657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.380636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.381461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.381840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.382118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.424406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.455514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.456981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.464392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa099e7-cecf-45a6-845c-fe92611c7734 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.488956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.501592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.502290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.502574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.502851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.503625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:12.503953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.505353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.518817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 538/1057] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:12.519954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.575116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 520/1058] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:30:12.580220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.626462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187ca61-347a-4165-8f8f-c5eaa6f667c3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.652428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.662134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.662936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.663249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.663524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.674310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.675824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.686215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b0e7-f031-4833-9846-17c93a2351f4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:12.692805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:12.693700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:12.694572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:12.694572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:12.694572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:12.694572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:12.694572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 539/1059] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:12.699717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.726123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.727298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.727582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.727858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.738917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.740412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.740864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:12.760907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:12.761848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:12.762746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:12.762746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:12.762746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:12.762746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:12.762746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 521/1060] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:12.765916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.816319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffd1c3-8bd2-4bc3-a084-54fac8935648 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.852652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.853716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.854042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.854538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.855482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:12.855930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.857471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.879517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 540/1061] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:12.880819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:12.883712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e867-c609-49f0-a45a-4a6a1ecbe68b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:12.909571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.922242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:12.922907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.923234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.923505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.924346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:12.924659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:12.926143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42802941-0019-4061-9f59-36a0849a8558 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:12.946149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 522/1062] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:12.946149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d77a-b926-4602-bd2e-9bbb369ff40d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.946149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d77a-b926-4602-bd2e-9bbb369ff40d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.946149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d77a-b926-4602-bd2e-9bbb369ff40d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.946149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 523/1063] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:12.947752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:12.948047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:12.948324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:12.968774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:12.970812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:12.993935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:12.998992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8b87-43fa-472d-be14-d7341227784b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.038971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.039828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.040149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.040376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.048553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.049212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 524/1064] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:13.050047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:30:13.065894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:13.067414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:13.068437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 541/1065] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:30:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:13.073227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.075215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.186914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc980af-1c72-4cd5-a0c8-c9fb374eb9c6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.202778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.209181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bff144-9a6a-469c-a1c8-9cbb2c17f410 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.226148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.226838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.227191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.227476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.228329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:13.228642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.230090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.243830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.244652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.244925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.245200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.247679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 525/1066] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:13.253759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.255590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:13.263093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:13.264097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:13.264970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 542/1067] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:13.270145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.289522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.351087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881df14-105f-4c49-8922-59987d484a65 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.373698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2b847-e9be-46e5-b168-f03b18f5822a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.385879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.386683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.386953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.387260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.397554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.399006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.406240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.407170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.407458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.407737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.408484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:13.408793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.410155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:13.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:13.417026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:13.418171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:13.418171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:13.418171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:13.418171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:13.418171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 526/1068] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:13.420766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 543/1069] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:13.422818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.426707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.449248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.509509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.526666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332cf82-ca3e-46ec-886e-c9abd4621198 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.529436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb6f31-a208-4308-a6d6-6d6770dfd6f4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.561007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.561684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.561968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.562067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.562242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.562949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.563060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:13.563382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.563777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.564059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] PATH_INFO: `/v3/users/aa160b566f22411e9082a996d36bf6da` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.564941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.576796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 527/1070] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:13.579793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] RBAC: Authorizing `identity:get_user(user_id=aa160b566f22411e9082a996d36bf6da)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.581299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.582420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.583007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 544/1071] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/users/aa160b566f22411e9082a996d36bf6da => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:13.588598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.665309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.684301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3871-9f40-47f9-804b-aad15ee54bf8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.685203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaea19b-0a9b-4dbe-83fa-5c7df0356d50 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.718180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.718905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.719334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.719534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.719950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.720048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.720236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.720511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.726173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.727671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.728403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:30:13.742475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:13.743596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:13.744586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 528/1072] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:13.749962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.789809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.791300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.849770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66847c-f6ba-47ee-921d-9513a985e44f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:13.884955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:13.885624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:13.885914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:13.886190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:13.886988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:13.887334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:13.888781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:13.889512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 545/1073] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:13.897701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.906014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 529/1074] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:13.911942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:13.914778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.936657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:13.997780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542f1bd-b6cc-4ffd-afed-42d6ce265c5a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:14.016648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd7b-e26e-481e-9bf4-b4d7bde7dc6d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:14.032512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:14.033378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:14.033658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:14.033937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:14.035134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:14.036603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:14.038319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 546/1075] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:13 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:14.044352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:14.051273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:14.052169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:14.052451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:14.052728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/33085f00b4eb426ea862c332e899e26e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:14.054980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=15d802693dbc490c890d922d838f856c, role_id=33085f00b4eb426ea862c332e899e26e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:14.056488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72a3bd-d084-4fc1-a22f-baac1f8b992e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:14.070570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 530/1076] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:30:13 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/15d802693dbc490c890d922d838f856c/roles/33085f00b4eb426ea862c332e899e26e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:30:14.152989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da41a-4c5f-47b0-82ac-6e53b710719c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:14.162075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:14.191431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:14.192105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:14.192384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:14.192654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:14.193485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=49151911107b41a99c66555cae2629ec, user.id=aa160b566f22411e9082a996d36bf6da) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:14.193793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:14.195290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455417ca-60b5-4ffc-a5e5-f67d06eed665 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:14.201408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 547/1077] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:14 2022] GET /identity/v3/role_assignments?user.id=aa160b566f22411e9082a996d36bf6da&scope.project.id=49151911107b41a99c66555cae2629ec&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:16.088454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb915-dd21-43d2-8e79-500c4e4d61be None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.088961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb915-dd21-43d2-8e79-500c4e4d61be None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.088961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fb915-dd21-43d2-8e79-500c4e4d61be None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.090316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 531/1078] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:16.096744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308be9-64c9-441f-94b2-fc67931d699f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.097216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308be9-64c9-441f-94b2-fc67931d699f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.097305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308be9-64c9-441f-94b2-fc67931d699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.114513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-72308be9-64c9-441f-94b2-fc67931d699f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:16.115731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-72308be9-64c9-441f-94b2-fc67931d699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:16.194358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 548/1079] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:16.217995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb915-dd21-43d2-8e79-500c4e4d61be None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.248635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858140-ee36-4879-8162-7345ecfb5089 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.248917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858140-ee36-4879-8162-7345ecfb5089 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.249191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858140-ee36-4879-8162-7345ecfb5089 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.250617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 549/1080] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:30:16.255716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.256007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.256271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.273213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:16.274374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:16.288231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.330516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fb915-dd21-43d2-8e79-500c4e4d61be None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.342578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 550/1081] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:16.363304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.364176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.364414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.364690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.365362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.375127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:16.383367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:16.384351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:16.385192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 532/1082] 10.222.0.65 () {56 vars in 1120 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:16.390402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.399271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.471938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254554-c782-4ba1-ab71-e464e9fc676a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.491063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b243fd-0896-4f33-b171-adf717674dcd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.512177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.512994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.513261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.513532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.515939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.524390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:16.527828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.528501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.528782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.529058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.529806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:16.530111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.531841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:16.532069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:16.533039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:16.533872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 551/1083] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:16.539401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.546667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 533/1084] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:16.547662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.628585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.638954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6ba12-5513-4f07-bcc6-0540a85dbfad None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.649893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f103eb-8584-4f2f-9881-0458fdb49701 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.672653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.673321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.673607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.673878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.674616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:16.674922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.676347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.684285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 552/1085] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:16.684789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.685891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.686176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.686451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.690224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.697746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.699238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:16.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:16.717770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:16.718658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:16.718658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:16.718658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:16.718658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:16.718658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 534/1086] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:16.723851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.760980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.800927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96624171-113c-4c53-b8f8-eeccb22bf39d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.827946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f56edb-1b19-4d47-b1f7-43917e336543 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.834810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.835758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.836038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.836310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] PATH_INFO: `/v3/users/aa160b566f22411e9082a996d36bf6da` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.837808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=aa160b566f22411e9082a996d36bf6da)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.839287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.841271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 553/1087] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/users/aa160b566f22411e9082a996d36bf6da => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:16.846694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.862888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.863603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.863889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.864169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.864897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:16.865225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.866605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.868221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.879106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 535/1088] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:16.884604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:16.945898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec3849-d2c9-403a-a424-1ddad5ac9bc0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.975056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:16.979154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:16.980049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:16.980330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:16.980605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:16.981444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893c57f-acad-4dfb-adb6-f1d9b02ad7e0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:16.981807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:16.983297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:16.985383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 554/1089] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:16.991018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.016889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.017698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.017986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.018264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.025325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.026772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:30:17.041726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:17.042793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:17.043806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 536/1090] 10.222.0.65 () {56 vars in 1093 bytes} [Fri Jun 24 09:30:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:30:17.048802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.077179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:17.089166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d784951-6a99-4d52-ace8-afda8341411d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:17.122991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.124277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.124563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.124840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec/users/aa160b566f22411e9082a996d36bf6da/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.127402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] RBAC: Authorizing `identity:create_grant(project_id=49151911107b41a99c66555cae2629ec, user_id=aa160b566f22411e9082a996d36bf6da, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.128894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.142808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 555/1091] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:30:16 2022] PUT /identity/v3/projects/49151911107b41a99c66555cae2629ec/users/aa160b566f22411e9082a996d36bf6da/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:30:17.147015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57bf4-c5ca-443f-8117-695d7ee43d1d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:17.185398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.186093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.186377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.186659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.187548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:17.187986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.189481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.209287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 537/1092] 10.222.0.65 () {56 vars in 1133 bytes} [Fri Jun 24 09:30:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:30:17.215385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.329322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef081c-9bbe-4201-a819-9f0ca01fa8ba None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:17.364770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.365429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.365711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.365990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.384143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:17.434084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.435606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.538561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 556/1093] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:17.544771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.562166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:17.652937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee79683-6891-4ded-9617-209c12b429ea None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:17.689014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.689734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.690019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.690290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.700707 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.702152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:17.720486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:17.721390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:17.722313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:30:17.722313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:17.722313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:30:17.722313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:17.722313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 538/1094] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:17.727635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.780576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:17.831839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7bd8-ed22-4bb4-a435-dccf267f7de6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:17.866803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:17.867522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:17.867808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:17.868088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:17.868807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:17.869120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:17.870878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:17.882470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 557/1095] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:17.888283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:17.957002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:17.992660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5387a-92cb-4757-a31d-0e5cf80d5103 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:18.028173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:18.028970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:18.029258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:18.029533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:18.036588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:18.038054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:30:18.051867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:18.052862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:30:18.053749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:18.055470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 539/1096] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:30:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:18.058865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:18.168840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b171-8a2d-4d8a-9d91-e48d184b56bf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:18.178104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:18.204478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:18.205167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:18.205446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:18.205704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:18.206491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:18.206797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:18.208311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f28b7-4b92-4d94-8c5b-a3a63ff34409 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:18.220789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 558/1097] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:18.227234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:18.331211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613497e-e792-4be7-84a2-4d92c8172766 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:18.361722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:18.370925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:18.371612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:18.371898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:18.372174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:18.373009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] RBAC: Adding query filter params (role.id=33085f00b4eb426ea862c332e899e26e, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=15d802693dbc490c890d922d838f856c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:18.373318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:18.374774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aba7d-6715-4f17-97bc-a9497eefe5a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:18.380733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 540/1098] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:18 2022] GET /identity/v3/role_assignments?user.id=15d802693dbc490c890d922d838f856c&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=33085f00b4eb426ea862c332e899e26e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:19.158090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0599793-184b-42ad-a471-239aec9376cd None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.158466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0599793-184b-42ad-a471-239aec9376cd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.158644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0599793-184b-42ad-a471-239aec9376cd None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.160092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 559/1099] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:19.165660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.165915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.166192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.184893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:19.186135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:19.258407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 541/1100] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:19.282861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0599793-184b-42ad-a471-239aec9376cd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:19.389184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0599793-184b-42ad-a471-239aec9376cd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:19.425950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:19.426661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.426938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.427244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.442225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:19.449878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:19.451068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:19.452008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 560/1101] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:19.457229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:19.565706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa5409-6050-42aa-a2ba-7c2a7efb7db7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:19.595714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:19.611690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:19.612367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.612652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.612928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.613672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:19.613978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:19.615516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:19.624240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 542/1102] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:19.632293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:19.669536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:19.744080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b49d2-cc10-4d3f-918e-91abe314e703 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:19.780439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:19.781344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.781659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.781908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] PATH_INFO: `/v3/users/aa160b566f22411e9082a996d36bf6da` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.783206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] RBAC: Authorizing `identity:get_user(user_id=aa160b566f22411e9082a996d36bf6da)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:19.784657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:19.786325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 561/1103] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:19 2022] GET /identity/v3/users/aa160b566f22411e9082a996d36bf6da => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:19.792347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:19.897030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28fc55-8c70-481b-b770-c847c0a4b499 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:19.933852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:19.934514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:19.934798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:19.935114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:19.994504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:20.014361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:20.016117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:20.119600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 543/1104] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:20.126787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:20.227486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:20.234060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9d19-2134-4524-9f70-a737975853cc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:20.280393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:20.281699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:20.282011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:20.282307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:20.283786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:20.285259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f5f11-feee-487f-bde1-98e73fd66db1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:20.286993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 562/1105] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:20 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:20.293495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:20.388218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:20.416836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b881-c3d1-4ae6-99f6-562a7cc0ad3d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:20.454658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:20.455476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:20.455671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:20.455952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:20.456794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=49151911107b41a99c66555cae2629ec, user.id=aa160b566f22411e9082a996d36bf6da) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:20.457111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:20.458571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffcc4c-281e-4829-aab2-9c5084d4cd6b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:20.471207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 544/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:20 2022] GET /identity/v3/role_assignments?user.id=aa160b566f22411e9082a996d36bf6da&scope.project.id=49151911107b41a99c66555cae2629ec&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:22.532055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05246e-e55a-45a8-b2fb-264db4a27bfc None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:22.532992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05246e-e55a-45a8-b2fb-264db4a27bfc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:22.532992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05246e-e55a-45a8-b2fb-264db4a27bfc None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:22.535253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 563/1107] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:22.544016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:22.544764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:22.544929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:22.570402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:22.571618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:22.652128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 545/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:22.679158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05246e-e55a-45a8-b2fb-264db4a27bfc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:22.790123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05246e-e55a-45a8-b2fb-264db4a27bfc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:22.827783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:22.828833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:22.828976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:22.829177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:22.830456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:22.831955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aeca7-9d68-4343-b53a-8684a278b63f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:22.833626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 564/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:22.842213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:22.924937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:22.949182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239c54e-6ac1-46a7-834c-c396c2ba9ee2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:22.988127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:22.988814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:22.989098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:22.989343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:22.990104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:22.991600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c984b1c-3516-4b3e-91db-7101e631ad7a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:23.016635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 546/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:30:22 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:25.086860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf25c1-6265-4d91-b980-c96cd669ea63 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.087490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf25c1-6265-4d91-b980-c96cd669ea63 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.087490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf25c1-6265-4d91-b980-c96cd669ea63 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.088759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 565/1111] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:25.095345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.096077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.096227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.118803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:25.120377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:25.192499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 547/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:25.216662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf25c1-6265-4d91-b980-c96cd669ea63 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:25.252561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:25.323709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf25c1-6265-4d91-b980-c96cd669ea63 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:25.359212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:25.360025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.360306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.360585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] PATH_INFO: `/v3/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.362070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] RBAC: Authorizing `identity:get_role(role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:25.363528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:25.364929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 566/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:25 2022] GET /identity/v3/roles/561eaf0348e94e0f924cd58981968818 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:25.371203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:25.433149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:25.475004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19129-9c95-4123-93c3-dee90b5885ce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:25.508217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:25.509102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.509386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.509661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] PATH_INFO: `/v3/users/55d30534189641ee9e91bdb898f6856c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.522804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] RBAC: Authorizing `identity:get_user(user_id=55d30534189641ee9e91bdb898f6856c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:25.524306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:25.525929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 548/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:25 2022] GET /identity/v3/users/55d30534189641ee9e91bdb898f6856c => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:25.531905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:25.634462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:25.639163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edb50e-17b7-490f-b793-7a7e282fa78a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:25.675473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:25.676240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.676531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.676805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.746089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:25.748023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:25.846404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:25.850542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 567/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:25.856401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:25.960002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff06a-9a05-4a16-a474-c0e3a38f275b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:25.980839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:25.995799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:25.996675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:25.996963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:25.997240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:25.998470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:26.000015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d7e17-ce05-4c42-95fe-e7f9d0f3622a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:26.001889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 549/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:25 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:26.008045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:26.115445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bbf98-957b-4b54-9e35-8ccd55175421 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:26.149798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:26.150455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:26.150744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:26.151016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:26.151871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=49151911107b41a99c66555cae2629ec, user.id=55d30534189641ee9e91bdb898f6856c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:26.152185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:26.153634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2bb00-a81b-4418-ba6c-eaca6e21b073 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:26.159743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 568/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:26 2022] GET /identity/v3/role_assignments?user.id=55d30534189641ee9e91bdb898f6856c&scope.project.id=49151911107b41a99c66555cae2629ec&role.id=561eaf0348e94e0f924cd58981968818 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:28.144228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d2e0-9d7b-43e4-9909-c8c8be17b90e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.144765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d2e0-9d7b-43e4-9909-c8c8be17b90e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.144854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168d2e0-9d7b-43e4-9909-c8c8be17b90e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.146883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 550/1118] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:28.155746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.156264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.156367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.181033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:28.182249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:28.249167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:28.262118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 569/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:28.286457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d2e0-9d7b-43e4-9909-c8c8be17b90e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:28.414350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168d2e0-9d7b-43e4-9909-c8c8be17b90e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:28.455109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:28.455996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.456278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.456555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] PATH_INFO: `/v3/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.458032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] RBAC: Authorizing `identity:get_role(role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:28.459459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:28.460826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 551/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:28 2022] GET /identity/v3/roles/561eaf0348e94e0f924cd58981968818 => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:28.466954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:28.592416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06fc02-29a4-415a-9eeb-876a4429b1f0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:28.647661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:28.648544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.648824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.649097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] PATH_INFO: `/v3/users/55d30534189641ee9e91bdb898f6856c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.650431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] RBAC: Authorizing `identity:get_user(user_id=55d30534189641ee9e91bdb898f6856c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:28.652091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:28.654656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 570/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:28 2022] GET /identity/v3/users/55d30534189641ee9e91bdb898f6856c => generated 286 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:28.662283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:28.731895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:28.785310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a1ee-f44e-4b5f-86d0-38f41e98bc13 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:28.818843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:28.819745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.820029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.820302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.821516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:28.822963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e5b33-cce1-43ab-acb0-d6968f772c1e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:28.824697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 552/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:28 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:28.832074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:28.914496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:28.956499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21111-b449-4b63-9dee-8550bab97130 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:28.995923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:28.996749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:28.997030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:28.997310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec/users/55d30534189641ee9e91bdb898f6856c/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:28.999745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] RBAC: Authorizing `identity:create_grant(project_id=49151911107b41a99c66555cae2629ec, user_id=55d30534189641ee9e91bdb898f6856c, role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:29.001204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368d2c4-c877-4f72-91ce-f17a23465611 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:29.017818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 571/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:30:28 2022] PUT /identity/v3/projects/49151911107b41a99c66555cae2629ec/users/55d30534189641ee9e91bdb898f6856c/roles/561eaf0348e94e0f924cd58981968818 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:30:30.998261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dab6e-9055-4c00-adf1-5582ba19c409 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:30.999095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dab6e-9055-4c00-adf1-5582ba19c409 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:30.999095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dab6e-9055-4c00-adf1-5582ba19c409 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:31.001677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 553/1124] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:31.015452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:31.016406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:31.016607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:31.046414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:31.048204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:31.134925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 572/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:31.170952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dab6e-9055-4c00-adf1-5582ba19c409 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:31.285835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dab6e-9055-4c00-adf1-5582ba19c409 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:31.322800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:31.341949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:31.343419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:31.343705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:31.344072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] PATH_INFO: `/v3/roles/561eaf0348e94e0f924cd58981968818` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:31.345753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] RBAC: Authorizing `identity:get_role(role_id=561eaf0348e94e0f924cd58981968818)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:31.347286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:31.349483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 554/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:31 2022] GET /identity/v3/roles/561eaf0348e94e0f924cd58981968818 => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:31.356451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:31.423051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:31.480048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6126b-51e0-418c-aaff-c9d7c9aaf135 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:31.516739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:31.517747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:31.518017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:31.518290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] PATH_INFO: `/v3/users/55d30534189641ee9e91bdb898f6856c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:31.520081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] RBAC: Authorizing `identity:get_user(user_id=55d30534189641ee9e91bdb898f6856c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:31.522342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:31.524993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 573/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:31 2022] GET /identity/v3/users/55d30534189641ee9e91bdb898f6856c => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:31.533378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:31.639152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8038a9-77e1-46ac-b72b-853acb5433e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:31.676318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:31.676974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:31.677256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:31.677530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:31.739178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:31.749852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:31.751620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:31.854666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 555/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:31.862675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:31.966605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b7467-bba4-4278-bbe5-bc809ac8e8b3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:31.986921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:32.005676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:32.006556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:32.006842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:32.007147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] PATH_INFO: `/v3/projects/49151911107b41a99c66555cae2629ec` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:32.008363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] RBAC: Authorizing `identity:get_project(project_id=49151911107b41a99c66555cae2629ec)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:32.009832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ceefb-b49d-4de9-83a1-6b405423d633 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:32.011590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 574/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:31 2022] GET /identity/v3/projects/49151911107b41a99c66555cae2629ec => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:32.019170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:32.129910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:32.139226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94bea5-7683-48cf-a456-5fb3f6b44f6e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:32.174472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:32.175155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:32.175446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:32.175728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:32.176557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] RBAC: Adding query filter params (role.id=561eaf0348e94e0f924cd58981968818, scope.project.id=49151911107b41a99c66555cae2629ec, user.id=55d30534189641ee9e91bdb898f6856c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:32.176872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:32.178320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d369a9-de7e-410f-a2bc-daaf4967a750 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:32.184148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 556/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:32 2022] GET /identity/v3/role_assignments?user.id=55d30534189641ee9e91bdb898f6856c&scope.project.id=49151911107b41a99c66555cae2629ec&role.id=561eaf0348e94e0f924cd58981968818 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:34.262313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64321f-4a25-47f0-adc0-29554f9f6fac None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:34.263232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64321f-4a25-47f0-adc0-29554f9f6fac None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:34.263232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64321f-4a25-47f0-adc0-29554f9f6fac None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:34.265567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 575/1131] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:34.271201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882818-5438-4b6b-bdf3-90004560834b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:34.271614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882818-5438-4b6b-bdf3-90004560834b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:34.271721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89882818-5438-4b6b-bdf3-90004560834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:34.291021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89882818-5438-4b6b-bdf3-90004560834b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:34.292260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89882818-5438-4b6b-bdf3-90004560834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:34.364490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 557/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:34.392494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64321f-4a25-47f0-adc0-29554f9f6fac None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:34.501735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64321f-4a25-47f0-adc0-29554f9f6fac None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:34.534800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:34.535575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:34.535857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:34.536129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:34.537316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:34.538781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a72f0d-3972-4d69-887e-37b3c357e0fe None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:34.540458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 576/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:34.548269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882818-5438-4b6b-bdf3-90004560834b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:34.653743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:34.670264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89882818-5438-4b6b-bdf3-90004560834b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:34.705401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:34.706063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:34.706344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:34.706619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:34.707385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:34.708829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728a044-566b-4348-9dac-eb8542ad4a58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:34.738256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 558/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:30:34 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:30:36.923939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88f69-d062-4add-aef1-acb79ea98fcb None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:36.924510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88f69-d062-4add-aef1-acb79ea98fcb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:36.924510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88f69-d062-4add-aef1-acb79ea98fcb None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:36.926718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 577/1135] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:36.932396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:36.932824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:36.933065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:36.955118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:36.956345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:37.040185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 559/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:37.066039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c88f69-d062-4add-aef1-acb79ea98fcb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:37.129958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:37.187295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c88f69-d062-4add-aef1-acb79ea98fcb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:37.226927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:37.228164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:37.228598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:37.229031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:37.230714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:37.232420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc86ad9-2899-4955-b7f2-807d70a7fd58 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:37.234381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 578/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:37.240099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:37.313085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:37.346401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0e1f-ebbf-4f60-8b25-3d18aa085407 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:37.383059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:37.383769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:37.384052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:37.384333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:37.385040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:37.386500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e8dfd-6623-4f5b-bcdf-86fcae244d3a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:37.412231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 560/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:30:37 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:30:39.544127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2bf65-df3b-441d-9f85-086ef6b1f77b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:39.544686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2bf65-df3b-441d-9f85-086ef6b1f77b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:39.544686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2bf65-df3b-441d-9f85-086ef6b1f77b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:39.546190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 579/1139] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:39.552368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:39.553011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:39.553110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:39.578812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:39.580042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:39.653237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 561/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:39.677980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2bf65-df3b-441d-9f85-086ef6b1f77b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:39.803122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2bf65-df3b-441d-9f85-086ef6b1f77b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:39.835254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:39.850404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:39.851325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:39.851610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:39.851883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:39.866091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:39.874211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:39.875533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:39.876596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 580/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:39.882334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:39.940504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:39.986251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0e798-dd36-4dce-8e72-261bb1171b3a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:40.023779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:40.024440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:40.024720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:40.024994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:40.025742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:40.026046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:40.027453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:40.036708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 562/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:40.042554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:40.151944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864cc2-5b72-481f-af67-e175d5bece6f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:40.188816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:40.189687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:40.189977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:40.190232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] PATH_INFO: `/v3/users/51c5667eb4d945c7b12e44f98f82b231` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:40.204139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] RBAC: Authorizing `identity:get_user(user_id=51c5667eb4d945c7b12e44f98f82b231)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:40.205617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:40.207363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 581/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:40 2022] GET /identity/v3/users/51c5667eb4d945c7b12e44f98f82b231 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:40.215097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:40.348160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a523e-6388-4da0-a025-7daabf483c58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:40.377165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:40.397120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:40.397787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:40.398074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:40.398353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:40.466579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:40.468379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:40.573913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 563/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:40.580561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:40.626869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:40.691447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ada9a-b798-4fc4-86d6-604d6c630671 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:40.727111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:40.727922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:40.728201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:40.728472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] PATH_INFO: `/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:40.729633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] RBAC: Authorizing `identity:get_project(project_id=4b59c3acbd6a4f9193d8b64338a9a87e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:40.731112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4625d-c786-4a91-95fa-bca26f0e4016 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:40.732802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 582/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:40 2022] GET /identity/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:40.740629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:40.760754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:40.853637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9fefd-d6e0-4ce5-9af8-2bed7a1d6101 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:40.907375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:40.908095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:40.908381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:40.908660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:40.909490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=4b59c3acbd6a4f9193d8b64338a9a87e, user.id=51c5667eb4d945c7b12e44f98f82b231) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:40.909803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:40.911266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf72bc7-84e4-47f4-b99d-fd769b8394d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:40.918081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 564/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:40 2022] GET /identity/v3/role_assignments?user.id=51c5667eb4d945c7b12e44f98f82b231&scope.project.id=4b59c3acbd6a4f9193d8b64338a9a87e&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:43.316122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7528-d25a-492b-9fb2-5442a0dc0539 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:43.316700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7528-d25a-492b-9fb2-5442a0dc0539 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:43.316816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c7528-d25a-492b-9fb2-5442a0dc0539 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:43.318234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 583/1147] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:43.328113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:43.328589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:43.329074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:43.355890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:43.357909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:43.445688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 565/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:43.471800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c7528-d25a-492b-9fb2-5442a0dc0539 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:43.592854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c7528-d25a-492b-9fb2-5442a0dc0539 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:43.612229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:43.639637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:43.640366 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:43.640650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:43.640922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:43.651389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:43.659139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:43.660166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:43.661230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 584/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:43.670680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:43.689754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:43.801721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd494-e4f5-4a39-836d-d37cee563c7b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:43.838308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:43.838989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:43.839320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:43.839610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:43.840364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:43.840678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:43.842076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:43.850755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 566/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:43.856683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:44.025628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f812b4-f878-44b0-86c5-a2d67be2cef6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:44.061077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:44.062637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:44.062917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:44.063242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] PATH_INFO: `/v3/users/51c5667eb4d945c7b12e44f98f82b231` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:44.064549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] RBAC: Authorizing `identity:get_user(user_id=51c5667eb4d945c7b12e44f98f82b231)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:44.066035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:44.067788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 585/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:43 2022] GET /identity/v3/users/51c5667eb4d945c7b12e44f98f82b231 => generated 286 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:44.073848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:44.161147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:44.194410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72351b0-8a19-450a-891c-b4bd016d97dc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:44.230174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:44.231048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:44.231380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:44.231647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] PATH_INFO: `/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:44.232834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] RBAC: Authorizing `identity:get_project(project_id=4b59c3acbd6a4f9193d8b64338a9a87e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:44.234489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb56c16-3f88-420f-ae69-0bd9e339ceba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:44.236320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 567/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:44 2022] GET /identity/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:44.242461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:44.317818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:44.401575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576d1b-0232-4c99-939d-3e84aff285f4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:44.443646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:44.444482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:44.444771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:44.445048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] PATH_INFO: `/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e/users/51c5667eb4d945c7b12e44f98f82b231/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:44.447912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b59c3acbd6a4f9193d8b64338a9a87e, user_id=51c5667eb4d945c7b12e44f98f82b231, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:44.450558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f7d33-4938-423b-9ee9-157d4a079940 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:44.468151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 586/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:30:44 2022] PUT /identity/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e/users/51c5667eb4d945c7b12e44f98f82b231/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:30:46.574322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da1b3-1ffa-47f6-b1e1-a4b9720e3c6a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:46.574893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da1b3-1ffa-47f6-b1e1-a4b9720e3c6a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:46.574893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da1b3-1ffa-47f6-b1e1-a4b9720e3c6a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:46.576246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 568/1154] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:46.581810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:46.582249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:46.582352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:46.603846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:46.605329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:46.692404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 587/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:46.718721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6da1b3-1ffa-47f6-b1e1-a4b9720e3c6a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:46.835249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6da1b3-1ffa-47f6-b1e1-a4b9720e3c6a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:46.873039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:46.874306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:46.874596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:46.874866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:46.883066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:46.886026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:30:46.894584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:46.895789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:30:46.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:30:46.901087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 569/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:46.902140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:46.930326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:47.018807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4b855-8cac-4739-a47a-33aa961658ad None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:47.061243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:47.061932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:47.062204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:47.062513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:47.063298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:47.063679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:47.065463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:47.073986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 588/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:47.080318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:47.199403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4847c-096b-4a64-82c7-ba8d8631a362 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:47.233552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:47.234424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:47.234712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:47.234990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] PATH_INFO: `/v3/users/51c5667eb4d945c7b12e44f98f82b231` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:47.236230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] RBAC: Authorizing `identity:get_user(user_id=51c5667eb4d945c7b12e44f98f82b231)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:47.237700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:47.239466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 570/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:30:47 2022] GET /identity/v3/users/51c5667eb4d945c7b12e44f98f82b231 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:47.245806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:47.349591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:47.360978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fdc-1076-4317-9084-6e4e762d0dea None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:47.415416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:47.416081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:47.416373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:47.416644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:47.490475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:47.491976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:47.584767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:47.596182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 589/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:47.602340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:47.673427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:47.709276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8892-5759-4b31-93b0-74ebec0ffb89 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:47.744303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:47.745221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:47.745570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:47.746159 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] PATH_INFO: `/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:47.747063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] RBAC: Authorizing `identity:get_project(project_id=4b59c3acbd6a4f9193d8b64338a9a87e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:47.748586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7e051-c43e-4ae6-8dd7-a957a71c5c9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:47.750299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 571/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:30:47 2022] GET /identity/v3/projects/4b59c3acbd6a4f9193d8b64338a9a87e => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:47.756705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:47.861028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c7042-6360-4c02-87ca-65c1035a8c90 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:47.896795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:47.897464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:47.897745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:47.898017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:47.898846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=4b59c3acbd6a4f9193d8b64338a9a87e, user.id=51c5667eb4d945c7b12e44f98f82b231) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:47.899213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:47.900620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be482b0-40a7-4bf5-84a5-a3fc5f76499e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:47.906795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 590/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:30:47 2022] GET /identity/v3/role_assignments?user.id=51c5667eb4d945c7b12e44f98f82b231&scope.project.id=4b59c3acbd6a4f9193d8b64338a9a87e&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:50.096072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e718-4c46-48d2-9cc7-2ed00f352b7e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:50.096867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e718-4c46-48d2-9cc7-2ed00f352b7e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:50.096867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e718-4c46-48d2-9cc7-2ed00f352b7e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:50.097890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 572/1162] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:50.103610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:50.104291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:50.104512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:50.138913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:50.140537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:50.200804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:50.220764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 591/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:30:50.244946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e718-4c46-48d2-9cc7-2ed00f352b7e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:50.366506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e718-4c46-48d2-9cc7-2ed00f352b7e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:50.409472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:50.410161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:50.410449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:50.410728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:50.415016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:50.495376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:50.496867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:50.604341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 573/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:30:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:30:50.611306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:50.731583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d28c1e-e391-4de6-ac6d-c42beb648930 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:50.781580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:50.782446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:50.782822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:50.783177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:50.791316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:50.792995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a79ab29c-142a-4e35-a10d-ee73b1c23bc4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:30:50.808416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:30:50.810346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:30:50.813445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 592/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:30:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:30:50.816941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:50.881108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:50.968684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be85a4-e279-4951-8a51-d6bf170ac668 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:51.011204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:51.012139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:51.012572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:51.012833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:51.013571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:30:51.013880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:51.017333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02488f-4361-4bc8-8204-ea5e8deb6203 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:51.025562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 574/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Fri Jun 24 09:30:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:53.102601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091031-5562-409f-9d89-3beac15396e0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:53.103576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091031-5562-409f-9d89-3beac15396e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:53.103576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091031-5562-409f-9d89-3beac15396e0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:53.105004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 593/1167] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:53.111455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d2d7-1980-4c5c-8af8-a859e97848d5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:53.111972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d2d7-1980-4c5c-8af8-a859e97848d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:53.112073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d2d7-1980-4c5c-8af8-a859e97848d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:53.133448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ef27d2d7-1980-4c5c-8af8-a859e97848d5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:53.134642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef27d2d7-1980-4c5c-8af8-a859e97848d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:53.208925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 575/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:53.240599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f091031-5562-409f-9d89-3beac15396e0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:53.294450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:53.359876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f091031-5562-409f-9d89-3beac15396e0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:53.395529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:53.396197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:53.396477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:53.396748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:53.397488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:53.398867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2d304-bdb1-4f35-8295-77357ac8ec96 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:53.418957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 594/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Fri Jun 24 09:30:53 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:30:55.546415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dd0e8-72fb-4789-8ee3-d1f7729e036c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:55.547366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dd0e8-72fb-4789-8ee3-d1f7729e036c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:55.547366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dd0e8-72fb-4789-8ee3-d1f7729e036c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:55.549598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 576/1170] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:55.558476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:55.559242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:55.559403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:55.579606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:55.580818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:55.655032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 595/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:55.678694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dd0e8-72fb-4789-8ee3-d1f7729e036c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:55.713804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:55.790521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dd0e8-72fb-4789-8ee3-d1f7729e036c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:55.837062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:55.837869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:55.838152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:55.838430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] PATH_INFO: `/v3/domains/14b411f31eb7440fa6b868e23b9084ee` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:55.849757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=14b411f31eb7440fa6b868e23b9084ee)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:55.851253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090042c-aecd-411e-ac31-d1b4696c6287 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:55.852896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 577/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Fri Jun 24 09:30:55 2022] GET /identity/v3/domains/14b411f31eb7440fa6b868e23b9084ee => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:55.859266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:55.895635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:55.972427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb319aa-ccaf-4e3f-9257-7ebb08d0636d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:56.013567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:56.014625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:56.014937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:56.015246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:56.015963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:56.017421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a6210-ccd1-4c43-98db-366992325a73 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:56.044374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 596/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:30:55 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:30:58.146063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d438f-cc86-41fc-907a-518f8caa51f1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:58.146593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d438f-cc86-41fc-907a-518f8caa51f1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:58.146593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d438f-cc86-41fc-907a-518f8caa51f1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:58.147934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 578/1174] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:30:58.154281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:58.154745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:58.154955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:58.173665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:30:58.174850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:30:58.251181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 597/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:30:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:30:58.278647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d438f-cc86-41fc-907a-518f8caa51f1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:58.377773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:58.389791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d438f-cc86-41fc-907a-518f8caa51f1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:58.427420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:58.429102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:58.429384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:58.429656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] PATH_INFO: `/v3/domains/14b411f31eb7440fa6b868e23b9084ee` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:58.431198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=14b411f31eb7440fa6b868e23b9084ee)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:58.437966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5fbdf-1d6c-4eb6-a5be-a2d261e897a3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:58.447743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 579/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Fri Jun 24 09:30:58 2022] GET /identity/v3/domains/14b411f31eb7440fa6b868e23b9084ee => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:30:58.455003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:30:58.474157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:30:58.573430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa4d30-4ee9-4b7f-841f-97484f359515 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:30:58.612563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:30:58.613249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:30:58.613534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:30:58.613797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:30:58.614515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:30:58.616004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f296f15-1e0e-4d02-8fb5-f171a80ca1f7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:30:58.644126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 598/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:30:58 2022] POST /identity/v3/users => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:31:00.848731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774c20-a16f-49a4-98ca-c3ffc882d6aa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:00.849384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774c20-a16f-49a4-98ca-c3ffc882d6aa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:00.850080 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774c20-a16f-49a4-98ca-c3ffc882d6aa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:00.854436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 580/1178] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:31:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:31:00.860658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:00.861375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:00.861375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:00.879956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:31:00.881192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:31:00.961921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 599/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:31:00.988151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d774c20-a16f-49a4-98ca-c3ffc882d6aa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:01.128412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d774c20-a16f-49a4-98ca-c3ffc882d6aa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:01.164569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:01.165457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:01.165736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:01.166006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:01.177236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:31:01.184717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:31:01.186037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:31:01.187067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 581/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:31:01.193344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:01.195729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:01.311815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d891c-9c95-435d-bd28-2d8128951f08 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:01.348876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:01.349558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:01.349842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:01.350123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:01.350878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:31:01.351210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:01.352616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:01.361527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 600/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:31:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:01.367599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:01.390534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:01.473685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c73f6-e15a-478e-9999-f84f09ab047b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:01.510407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:01.511332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:01.511618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:01.511893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] PATH_INFO: `/v3/users/d882e8f175074a688818499aaaf38dd7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:01.526983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] RBAC: Authorizing `identity:get_user(user_id=d882e8f175074a688818499aaaf38dd7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:01.528503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:01.530424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 582/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:31:01 2022] GET /identity/v3/users/d882e8f175074a688818499aaaf38dd7 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:01.538756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:01.604831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:01.645319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1cb7b-aeb2-42ea-9f87-b5fff0d625d9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:01.679794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:01.680446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:01.680726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:01.681000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:01.750951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:01.752473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:01.815764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:01.858496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 601/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:31:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:31:01.864665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:01.947365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:01.972047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abf489-d4c3-49e7-8f83-7aa825a572a0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:02.007821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:02.008692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:02.008973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:02.009262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] PATH_INFO: `/v3/projects/d76702b60d3c44379bd80a591f9e8282` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:02.010415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] RBAC: Authorizing `identity:get_project(project_id=d76702b60d3c44379bd80a591f9e8282)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:02.011902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a081214-0253-44af-8853-eefebab6db26 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:02.014397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 583/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:31:01 2022] GET /identity/v3/projects/d76702b60d3c44379bd80a591f9e8282 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:02.020438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:02.138863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb65327-89a6-46b2-afc6-036a9a74221e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:02.175927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:02.176598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:02.176876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:02.177155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:02.177987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=d76702b60d3c44379bd80a591f9e8282, user.id=d882e8f175074a688818499aaaf38dd7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:31:02.178297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:02.179798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb28af0-192f-465e-9b98-b14fb0c2acee None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:02.186331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 602/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:31:02 2022] GET /identity/v3/role_assignments?user.id=d882e8f175074a688818499aaaf38dd7&scope.project.id=d76702b60d3c44379bd80a591f9e8282&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:04.218787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2414f7-62eb-4ff1-b29d-a18075c08173 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.219550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2414f7-62eb-4ff1-b29d-a18075c08173 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.219550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2414f7-62eb-4ff1-b29d-a18075c08173 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.220744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 584/1186] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:31:04.226466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.226940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.227032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.245635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:31:04.246846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:31:04.273607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:04.322388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 603/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:31:04.346071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2414f7-62eb-4ff1-b29d-a18075c08173 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:04.461301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2414f7-62eb-4ff1-b29d-a18075c08173 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:04.497247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:04.498018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.498392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.498671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.509418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:31:04.517145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:31:04.518639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:31:04.520478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 585/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:31:04.526330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:04.630596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7256f-196b-40e8-9c32-2a93d990f1bc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:04.665976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:04.666652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.666938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.667234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.667982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:31:04.668284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:04.670162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:04.678971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 604/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:04.686054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:04.692452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:04.808630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e3590-f911-4368-beca-3f85c4223262 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:04.844624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:04.845498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.845785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.846057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] PATH_INFO: `/v3/users/d882e8f175074a688818499aaaf38dd7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.847321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] RBAC: Authorizing `identity:get_user(user_id=d882e8f175074a688818499aaaf38dd7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:04.848771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:04.850390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 586/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:31:04 2022] GET /identity/v3/users/d882e8f175074a688818499aaaf38dd7 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:04.856565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:04.880858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:04.959848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6c476-f317-4a30-b1b0-aec2e8799dc2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:04.994784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:04.995681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:04.995960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:04.996230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] PATH_INFO: `/v3/projects/d76702b60d3c44379bd80a591f9e8282` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:04.997625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] RBAC: Authorizing `identity:get_project(project_id=d76702b60d3c44379bd80a591f9e8282)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:04.999301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5135c-e0a3-4f51-a4aa-28d476bc4083 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:05.001146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 605/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:31:04 2022] GET /identity/v3/projects/d76702b60d3c44379bd80a591f9e8282 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:05.007500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:05.083876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:05.116816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075b671-7077-45e1-a27c-992e01351ecd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:05.152676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:05.153579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:05.153954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:05.154208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] PATH_INFO: `/v3/projects/d76702b60d3c44379bd80a591f9e8282/users/d882e8f175074a688818499aaaf38dd7/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:05.156231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] RBAC: Authorizing `identity:create_grant(project_id=d76702b60d3c44379bd80a591f9e8282, user_id=d882e8f175074a688818499aaaf38dd7, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:05.157709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798a877-976a-45d6-ad1b-7b23a879d94c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:05.176985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 587/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:31:05 2022] PUT /identity/v3/projects/d76702b60d3c44379bd80a591f9e8282/users/d882e8f175074a688818499aaaf38dd7/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:31:07.251512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0dbb-d0ea-4cf3-984d-d0d90cdf8572 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:07.252103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0dbb-d0ea-4cf3-984d-d0d90cdf8572 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:07.252436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a0dbb-d0ea-4cf3-984d-d0d90cdf8572 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:07.254587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 606/1193] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:31:07.262515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:07.263356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:07.263537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:07.281625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:31:07.282821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:31:07.357848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 588/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:31:07.383667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a0dbb-d0ea-4cf3-984d-d0d90cdf8572 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:07.478763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:07.494467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a0dbb-d0ea-4cf3-984d-d0d90cdf8572 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:07.532881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:07.533600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:07.533878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:07.534148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:07.546455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:31:07.553974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:31:07.555004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:31:07.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 607/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:31:07.562173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:07.631469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:07.673542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e3f7-fde1-4aac-8120-8610318a4b3a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:07.708610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:07.709274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:07.709553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:07.709806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:07.710560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:31:07.710859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:07.712278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:07.721806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 589/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:31:07.728060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:07.837580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cde58-52f7-484e-8f3b-2c6af59d4de0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:07.873575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:07.874457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:07.874738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:07.875012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] PATH_INFO: `/v3/users/d882e8f175074a688818499aaaf38dd7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:07.876293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] RBAC: Authorizing `identity:get_user(user_id=d882e8f175074a688818499aaaf38dd7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:07.877776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:07.879493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 608/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Fri Jun 24 09:31:07 2022] GET /identity/v3/users/d882e8f175074a688818499aaaf38dd7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:07.885505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:07.990000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969b485-d84f-4155-ab47-0a7638cb018d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:08.014697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:08.026006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:08.026663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:08.026946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:08.027262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:08.098133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:08.099539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:08.202865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 590/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:31:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:31:08.209110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:08.228844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:08.315591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f47ad-3149-4a65-b67a-5345aead5510 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:08.353141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:08.353903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:08.354211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:08.354447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] PATH_INFO: `/v3/projects/d76702b60d3c44379bd80a591f9e8282` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:08.355685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] RBAC: Authorizing `identity:get_project(project_id=d76702b60d3c44379bd80a591f9e8282)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:08.357263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6ebc-dd45-45b4-8e4f-4934ca814d27 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:08.358965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 609/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Fri Jun 24 09:31:08 2022] GET /identity/v3/projects/d76702b60d3c44379bd80a591f9e8282 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:08.365316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:31:08.391840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:08.473375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e3140-33fd-4a52-ab44-69544121ca6d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:31:08.509551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:31:08.510213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:31:08.510501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:31:08.510782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:31:08.511650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=d76702b60d3c44379bd80a591f9e8282, user.id=d882e8f175074a688818499aaaf38dd7) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:31:08.511974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:31:08.513491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebdb6-6836-4f0f-b9ad-e798e7d3c670 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:31:08.528040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 591/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:31:08 2022] GET /identity/v3/role_assignments?user.id=d882e8f175074a688818499aaaf38dd7&scope.project.id=d76702b60d3c44379bd80a591f9e8282&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:31:18.348258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=58 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:31:18.517267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=56 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:04.472219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38915798-3605-4d55-8d85-24e766658544 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:04.472962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38915798-3605-4d55-8d85-24e766658544 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:04.472962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38915798-3605-4d55-8d85-24e766658544 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:04.474138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 610/1201] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:04.480326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:04.480985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:04.480985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:04.501120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:04.502418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:04.594573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 592/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:04.622872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38915798-3605-4d55-8d85-24e766658544 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:04.754672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38915798-3605-4d55-8d85-24e766658544 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:04.803436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:04.804580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:04.804922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:04.805214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:04.819510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:04.821583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:04.845032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:04.846540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:04.847952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:04.847952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:04.847952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:04.847952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:04.852928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:04.856669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 611/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:04.998675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb7e9-6523-482f-88b4-1ffb3cdc02e7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:05.031137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:05.038403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:05.039113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:05.039402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:05.041798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:05.041798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:05.041798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:05.042655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d6232-d330-4d31-9759-eafc7e34df61 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:05.055384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 593/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:05.061629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:05.132387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:05.201205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb28427-ed00-4d6c-85cf-ea61c2ff8702 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:05.244058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:05.245151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:05.245427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:05.245800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:05.246620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:05.248238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0426a73-8489-4bf9-bd11-26f7d7b7d011 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:05.283194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 612/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Fri Jun 24 09:32:05 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:32:07.397020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff77e3-d1ab-4abe-9164-63595b667a04 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:07.397590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff77e3-d1ab-4abe-9164-63595b667a04 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:07.397590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff77e3-d1ab-4abe-9164-63595b667a04 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:07.399027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 594/1206] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:07.405323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:07.405803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:07.405924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:07.425185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:07.426542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:07.514293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 613/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:07.537876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff77e3-d1ab-4abe-9164-63595b667a04 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:07.665103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff77e3-d1ab-4abe-9164-63595b667a04 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:07.708924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:07.709759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:07.710040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:07.710314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:07.723402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:07.732177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:07.733385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:07.734318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 595/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:32:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:32:07.742712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:07.796765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:07.864751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ca5b-4a35-4c64-b5fd-60e8738e4d45 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:07.908282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:07.909426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:07.909816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:07.910178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:07.911095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:07.911620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:07.913730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:07.924070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 614/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:32:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:07.929885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:07.985949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:08.047946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b499a1c-271b-4bca-96b1-b32c14522249 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:08.087699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:08.088561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:08.088843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:08.089129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:08.100556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:08.102226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:08.121866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:08.123023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:08.124203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:08.124203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:08.124203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:08.124203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:08.127228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 596/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:32:08.129036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:08.235336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:08.243819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2144fa2-7fc2-4021-8b1a-f35e27ae1bbf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:08.278913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:08.279627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:08.279915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:08.280196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:08.281062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:08.281515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:08.283486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:08.301101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 615/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:08.306647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:08.427267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c67a23-8961-40c1-8c3e-3287631b1bb2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:08.429066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:08.466228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:08.467308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:08.467713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:08.468152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:08.477774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:08.479468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:08.496862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:08.498265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:08.499574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:08.503062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 597/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:08.503917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:08.623283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cea3c-ac11-4f03-b3e5-57a13790bcf1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:08.662065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:08.662829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:08.663198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:08.663514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:08.664514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:08.664839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:08.668213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:08.687852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 616/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:08.693788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:08.828524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89708a97-be09-4b29-a3b6-44b9d951cf6b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:08.877343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:08.878101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:08.878383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:08.878654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:08.919887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:08.969894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:08.971526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:09.099149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 598/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:32:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:32:09.101246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:09.124856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:09.223014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68c6d4-5a17-45f1-92da-1de4fc5d73a5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:09.271083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:09.271947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:09.272278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:09.272591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:09.283438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:09.284930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:09.303054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:09.304089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:09.305019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:09.305019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:09.305019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:09.305019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:09.305019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 617/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:32:09.310388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:09.392598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:09.441769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2391c-b0d0-4f35-a141-0d1b240777d6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:09.481984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:09.481984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:09.481984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:09.482390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:09.482804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:09.483218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:09.485212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:09.504789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 599/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:09.506426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:09.590547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:09.616519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c5f5d-2c08-45a4-bad4-857a598d4439 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:09.662394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:09.663309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:09.663536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:09.663828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:09.672149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:09.673752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:09.688782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:09.696515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:09.697518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 618/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:09.698871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:09.840534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dcaa8-eab6-4ff8-8eaf-6eb28c193fed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:09.869206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:09.885551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:09.886597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:09.887030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:09.887554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:09.888934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:09.889466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:09.892014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0358ff-bee4-4d6e-b062-0da1cf0565a8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:09.908555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 600/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:09.915829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:10.053544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422bdf-1372-42cc-a21f-5a5388d762e7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:10.085807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:10.093084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:10.093854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:10.094138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:10.094414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:10.095367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=e945abf5978842a79cfaf92c479542a7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:10.095685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:10.097188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1ff17-cf39-42f1-82df-5afb3eccc7eb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:10.106134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 619/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:32:09 2022] GET /identity/v3/role_assignments?user.id=e945abf5978842a79cfaf92c479542a7&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:12.292694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ce180-3610-4d08-9d1e-e1a80b04aa58 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:12.293246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ce180-3610-4d08-9d1e-e1a80b04aa58 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:12.293246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ce180-3610-4d08-9d1e-e1a80b04aa58 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:12.294639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 601/1220] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:12.299957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:12.300436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:12.300523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:12.317816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:12.319002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:12.390332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 620/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:12.413341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ce180-3610-4d08-9d1e-e1a80b04aa58 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:12.541755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ce180-3610-4d08-9d1e-e1a80b04aa58 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:12.583582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:12.584657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:12.585054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:12.585442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:12.598388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:12.607588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:12.608600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:12.615225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 602/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:32:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:12.615933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:12.725318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bc066-79c5-4985-b687-70434d2b170a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:12.754970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:12.767491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:12.768189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:12.768475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:12.768818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:12.769654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:12.769999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:12.771548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:12.780704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 621/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:32:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:12.786720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:12.846198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:12.923445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2859-af3c-499d-a2c2-c9070e0ffd77 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:12.965571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:12.966381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:12.966659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:12.966936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:12.978334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:12.980451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:13.000346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:13.001467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:13.002504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:13.002504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:13.002504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:13.002504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:13.002504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 603/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:13.008703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.125805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88b742-9117-4f01-9945-19d67567ae51 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:13.159938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:13.160755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:13.161174 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:13.161515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:13.162550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:13.162881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:13.164671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.179983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 622/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:13.186653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.297984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:13.305672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374333b9-23c7-4ce4-9d73-4a12a047ebed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:13.347318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:13.348366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:13.348685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:13.349025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:13.357518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:13.359285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:13.378929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:13.379942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:13.380885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 604/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:13.387193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.437835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:13.504237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd6291-7538-4772-8e61-a0f102df40d2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:13.544873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:13.545708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:13.546003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:13.546364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:13.547404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:13.547738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:13.549653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.567710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 623/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:13.574331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.687288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ee4dd-5f43-43a2-b181-57a939431dcb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:13.724628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:13.725405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:13.725688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:13.725962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:13.737454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:13.739641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:13.763274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:13.764320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:13.765207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:13.765207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:13.765207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:13.765207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:13.765207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 605/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:13.771243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.881237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193865-40a7-4151-8f93-f6f6548a979f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:13.883602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:13.919105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:13.919892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:13.920175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:13.920445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:13.921158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:13.921463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:13.922858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:13.935426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 624/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:13.941476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:13.950920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:14.055875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979cd58-009e-4419-82df-6362229de2e4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:14.092320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:14.093416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:14.093713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:14.094093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:14.102172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:14.103717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:14.120824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:14.121748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:14.122643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 606/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:14.128707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:14.245512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f53463-844d-4cd1-aea6-48be7bc747e8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:14.290460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:14.291263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:14.291831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:14.292070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:14.292798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:14.293109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:14.294614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd01fbe-c503-4eec-ac29-c9a40fa218e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:14.307672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 625/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:14.315351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:14.398810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:14.438771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54056c7-d502-4a11-a210-3d4082682860 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:14.479862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:14.480857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:14.481208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:14.481644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/e945abf5978842a79cfaf92c479542a7/roles/6e266b191bf040788d216f4ff72c295a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:14.503600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=e945abf5978842a79cfaf92c479542a7, role_id=6e266b191bf040788d216f4ff72c295a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:14.505520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073edc1-93a9-4d21-a322-2d93aef46bcc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:14.521964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 607/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:32:14 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/e945abf5978842a79cfaf92c479542a7/roles/6e266b191bf040788d216f4ff72c295a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:32:16.587647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4be4a-9088-4737-b2f2-288a2fad6846 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:16.588684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4be4a-9088-4737-b2f2-288a2fad6846 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:16.588684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4be4a-9088-4737-b2f2-288a2fad6846 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:16.589872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 626/1233] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:16.595310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:16.595785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:16.595866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:16.613066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:16.614614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:16.683663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 608/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:16.707029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4be4a-9088-4737-b2f2-288a2fad6846 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:16.736952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:16.824777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4be4a-9088-4737-b2f2-288a2fad6846 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:16.865168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:16.865952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:16.866238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:16.866545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:16.877722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:16.886121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:16.887401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:16.888313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 627/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Fri Jun 24 09:32:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:16.894247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:16.943058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:16.998779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b3952-422d-4031-be72-2274a87a9220 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.033132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.033808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.034086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.034411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.035121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:17.035426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.036801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:17.045195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 609/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Fri Jun 24 09:32:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:17.050998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:17.140247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:17.158472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90913bf4-0627-4cb2-a1d7-eac3ef18f4a1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.195041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.195885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.196165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.196439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.207520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.209058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:17.226443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:17.227567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:17.228581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:17.228581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:17.228581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:17.228581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:17.228581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 628/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:17.234457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:17.340717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156f4bc-1ea7-47ec-93a8-ba713eb1abdd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.352239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:17.382982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.383683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.383969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.384246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.385044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:17.385499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.386887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:17.400051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 610/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:17.406015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:17.515858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64bff-8075-4d30-9f70-3247904515cb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.553912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.554818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.555130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.555410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.558682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:17.564826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.566345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:17.581317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:17.582493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:17.583465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 629/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:17.588943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:17.697433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0830d-f06e-4d41-b5c7-59a3a14dcbd3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.734542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.735250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.735541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.735889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.736653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:17.736962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.738403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:17.758992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 611/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:17.766359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:17.882234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580a227-ecf7-4c74-af39-f6d7c3ebe68d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:17.919394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:17.920119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:17.920453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:17.920740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:17.987151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:17.996452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:17.997939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:18.102588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 630/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:32:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:32:18.108875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:18.140399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:18.223567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb843040-2ab3-49bc-b9ae-55e0b461e958 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:18.260768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:18.261614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:18.261854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:18.262145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:18.272457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:18.273961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:18.292597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:18.293870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:18.294914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:18.294914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:18.294914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:18.294914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:18.294914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 612/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:18.299679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:18.407116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:18.413307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173e58-d523-406a-858b-2fba4a5f944a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:18.451839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:18.452518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:18.452799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:18.453056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:18.453782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:18.454088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:18.455536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:18.469091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 631/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:18.475024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:18.565683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:18.588143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c6053-3179-4a81-91cd-c5ccb6fe9601 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:18.625036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:18.625903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:18.626161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:18.626448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:18.633359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:18.634874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:18.649463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:18.650613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:18.656439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 613/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:18.657158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:18.774488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a247-90d2-4d78-9e75-6fe26ef8c467 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:18.815872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:18.816486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:18.816760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:18.817039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:18.817822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:18.818288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:18.819962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e30f-ae7c-45b8-aaa6-0baab781d80b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:18.833843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 632/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:18.840960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:18.955060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82198307-6377-4475-a9e4-200bc19bb390 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:18.991647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:18.992453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:18.992814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:18.993090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:18.993980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] RBAC: Adding query filter params (role.id=6e266b191bf040788d216f4ff72c295a, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=e945abf5978842a79cfaf92c479542a7) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:18.994613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:18.995875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586538b7-0d33-4a71-a6b2-a62b78b7b9db None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:19.003125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 614/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:32:18 2022] GET /identity/v3/role_assignments?user.id=e945abf5978842a79cfaf92c479542a7&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=6e266b191bf040788d216f4ff72c295a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:21.301825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0dcf-c490-40fe-b477-35b05de2b771 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:21.302408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0dcf-c490-40fe-b477-35b05de2b771 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:21.302408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e0dcf-c490-40fe-b477-35b05de2b771 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:21.303828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 633/1247] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:21.313008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:21.313459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:21.313585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:21.319176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:21.334447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:21.336138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:21.419794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 615/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:21.446154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e0dcf-c490-40fe-b477-35b05de2b771 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:21.467636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:21.572849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e0dcf-c490-40fe-b477-35b05de2b771 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:21.610419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:21.611332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:21.611625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:21.611904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:21.623850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:21.631922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:21.632934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:21.633756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 634/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:21.640208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:21.748518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caeb9e-e91d-46a2-80a3-2723ac27eeaa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:21.788775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:21.794950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:21.795668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:21.795958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:21.796238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:21.797006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:21.797317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:21.798709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:21.808466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 616/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:21.814405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:21.905715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:21.930994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cfc64-6021-4554-9c5e-35c23d3d47f8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:21.970291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:21.971296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:21.971580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:21.971871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:21.984224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:21.985737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:22.004370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:22.005556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:22.010202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:22.010202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:22.010202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:22.010202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:22.010202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 635/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:22.011993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:22.120182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f002c3-e3e9-4769-b164-a59aaf06f219 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:22.155616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:22.156378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:22.156595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:22.156916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:22.157602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:22.157907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:22.159333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:22.172084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 617/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:22.178108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:22.301640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79d089-db7e-4341-aff7-ded9f4fce617 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:22.332308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:22.342384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:22.343392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:22.343748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:22.344099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:22.356073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:22.358356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:22.376287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:22.377469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:22.379192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 636/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:22.384926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:22.459189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:22.514358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb290296-0eb2-46a0-a107-58c3a999431b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:22.561068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:22.562071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:22.562392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:22.562810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:22.563993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:22.564452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:22.566419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:22.586677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 618/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:22.593509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:22.718513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927420-40af-4d15-939d-b0571ede973b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:22.759151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:22.760446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:22.760730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:22.761007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:22.827352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:22.843547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:22.845429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:22.964306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 637/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:32:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 09:32:22.970308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:23.087097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71d2ff-b6ff-42b1-a8be-cfc48ade6d1b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:23.102601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:23.128886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:23.129793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:23.130149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:23.130493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:23.142890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:23.144554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:23.168591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:23.169937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:23.177104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:23.177104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:23.177104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:23.177104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:23.177104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 619/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:23.177552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:23.288350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:23.302642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede88565-bb30-4b09-a984-30f248e804a0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:23.341926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:23.342643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:23.342943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:23.343246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:23.344074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:23.344382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:23.345985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:23.359402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 638/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:23.366204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:23.486813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65400fe-5e23-434a-ad5c-e90d6170cf03 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:23.529744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:23.530582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:23.530871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:23.531151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:23.538999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:23.541101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:23.559419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:23.565832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:23.566898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 620/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:23.567463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:23.676222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e446b-d532-4996-ae0b-11ff7ef914e5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:23.714429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:23.715153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:23.715440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:23.715717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:23.716512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:23.716829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:23.718289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf15c3-eb48-40b8-8648-a0213f298486 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:23.732396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 639/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:23.739615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:23.775283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:23.861089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82ee8-d801-48e7-acab-941b9bac5580 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:23.900475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:23.901265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:23.901552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:23.901890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:23.902869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=e945abf5978842a79cfaf92c479542a7) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:23.903297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:23.904982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e107-f26f-4690-a53b-cea9fd41e7a0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:23.911561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 621/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:32:23 2022] GET /identity/v3/role_assignments?user.id=e945abf5978842a79cfaf92c479542a7&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:25.987781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8de4d-a8d9-4a25-8f83-7699ee0c3b93 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:25.988728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8de4d-a8d9-4a25-8f83-7699ee0c3b93 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:25.988728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8de4d-a8d9-4a25-8f83-7699ee0c3b93 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:25.990099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 640/1261] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:25.996659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:25.997071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:25.997171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.019139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:26.020332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:26.089718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 622/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:26.124673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8de4d-a8d9-4a25-8f83-7699ee0c3b93 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:26.141686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:26.246013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8de4d-a8d9-4a25-8f83-7699ee0c3b93 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:26.281510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:26.282239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:26.282526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:26.282803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.293629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:26.301251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:26.302292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:26.303345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 641/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:32:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:26.309047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:26.328944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:26.419796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e95c5-a0c0-447d-ac2c-06c247dd76e5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:26.459026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:26.459727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:26.460013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:26.460285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.461037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:26.461348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:26.462774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:26.470844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 623/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:26.477110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:26.552632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:26.586872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b6f42-893d-40ef-b6a5-972d594ebfa5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:26.622157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:26.622955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:26.623342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:26.623616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.634652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:26.636195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:26.653456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:26.654497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:26.655613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:26.655613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:26.655613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:26.655613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:26.659470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 642/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:26.660555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:26.742412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:26.769632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3d256-8c0a-49b4-abf1-e4780b0e2bf9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:26.805275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:26.805870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:26.806235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:26.806673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.807427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:26.807734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:26.809107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:26.820760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 624/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:26.829699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:26.936314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102de80-7155-4698-899f-548827ed9f71 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:26.957875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:26.973850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:26.974651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:26.974935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:26.975188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:26.982840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:26.984329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:26.999228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:27.000720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:27.001888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 643/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:27.008532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.113494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac94fb-b999-4902-9627-f735e1360ccc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:27.150364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:27.151032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:27.151374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:27.151614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:27.152449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:27.152758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:27.154198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:27.163054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:27.176709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 625/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:27.182600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.297833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef92b19-4733-4e4a-861e-7392b4bb0ea9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:27.333388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:27.334177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:27.334453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:27.334722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:27.345430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:27.346877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:27.364228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:27.365236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:27.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:27.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:27.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:27.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:27.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 644/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:27.371309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.479346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50b31e-ede0-4ddb-b3a3-49e2c487ca50 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:27.516286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:27.516962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:27.517238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:27.517511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:27.518230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:27.518535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:27.519948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:27.532919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 626/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:27.538941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.573715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:27.647615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f639182-4977-4093-9751-4ca7ebd2a315 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:27.684793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:27.685637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:27.685924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:27.686195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:27.693597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:27.695065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:27.715118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:27.716136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:27.717210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 645/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:27.722356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.791712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:27.842472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf782-f941-43a5-bd81-f90d1d2662fd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:27.878960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:27.879640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:27.879922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:27.880195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:27.880970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:27.881277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:27.882708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c6c57-f2ce-4b67-83e4-67b0c8704f91 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:27.895473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 627/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:27.902089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:27.999675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:28.009900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9fab7-766b-4b8f-8775-a99673562026 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:28.047693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:28.048579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:28.048856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:28.049130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] PATH_INFO: `/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/e945abf5978842a79cfaf92c479542a7/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:28.051305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f353bca7604872a4e99359eaeef3ff, user_id=e945abf5978842a79cfaf92c479542a7, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:28.052770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b30bc-87d1-4958-b349-81ec96e04830 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:28.067860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 646/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Fri Jun 24 09:32:27 2022] PUT /identity/v3/projects/52f353bca7604872a4e99359eaeef3ff/users/e945abf5978842a79cfaf92c479542a7/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:32:30.095571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc5e90-a878-4c09-8fa8-d46bdfb1df20 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.096097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc5e90-a878-4c09-8fa8-d46bdfb1df20 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.096097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc5e90-a878-4c09-8fa8-d46bdfb1df20 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.097447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 628/1274] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:30.103772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.104526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.104679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.129013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:30.130214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:30.206851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 647/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:30.230839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc5e90-a878-4c09-8fa8-d46bdfb1df20 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:30.340834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc5e90-a878-4c09-8fa8-d46bdfb1df20 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:30.358629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:30.379684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:30.380483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.380775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.381056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.392300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 09:32:30.399950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:30.401003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:30.402100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 629/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Fri Jun 24 09:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:30.407277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:30.514072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:30.532096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49539c7c-98ff-402a-b88c-22ebd94a92f8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:30.572924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:30.573624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.573904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.574178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.574927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:30.575267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:30.576660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:30.584629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 648/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Fri Jun 24 09:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:30.591222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:30.709645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144d866-53c1-4258-9dc7-9d22bab46659 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:30.748002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:30.748810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.749095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.749370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.761460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:30.762932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:30.780788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:30.781740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:30.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:30.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:30.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:30.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:30.782743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 630/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:30.790426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:30.896559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e82bdf-26d2-4e8c-ad1c-51ef7ecfc727 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:30.932132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:30.932790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:30.933080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:30.933360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:30.934087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:30.934398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:30.935819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:30.949253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 649/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:30.955193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:30.988553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:31.064435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3f595-c7d6-4150-9a58-a7ffb42039ac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:31.100143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:31.100939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:31.101214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:31.101484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:31.110458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:31.111938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 09:32:31.127110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:31.128195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:31.129484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 631/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:31.134382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:31.138315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:31.236957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc52b5-a7cc-4e59-b124-9c93368d4a67 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:31.273160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:31.273759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:31.274047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:31.274317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:31.275194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:31.275499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:31.276939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:31.293744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 650/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Fri Jun 24 09:32:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:31.300319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:31.403128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:31.421056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a47e1a-6e24-4ae5-94ef-a90afaeb18b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:31.458886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:31.459575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:31.459862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:31.460174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:31.542236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:31.543666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:31.635454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:31.651547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 632/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Fri Jun 24 09:32:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 09:32:31.660702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:31.734796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:31.777143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ca97-184c-4b11-9886-7cc2c2fcd282 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:31.815690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:31.816431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:31.816717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:31.816994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:31.827788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:31.829278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 09:32:31.846855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:31.847858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:31.852637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 09:32:31.852637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 09:32:31.852637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 09:32:31.852637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:31.852637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 651/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:31.855493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:31.962327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49cb77-8e65-4681-aff1-592beb07cf64 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:31.999458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:32.000106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:32.000388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:32.000661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:32.001375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:32.001685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:32.003055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:32.014722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 633/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Fri Jun 24 09:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:32.021475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:32.133223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281bb2-1f79-45de-9038-3d625b2bf1d2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:32.147089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:32.170593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:32.171444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:32.171732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:32.172008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:32.179106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:32.180672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 09:32:32.194361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:32.195408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Jun 24 09:32:32.196510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 652/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Fri Jun 24 09:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:32.203540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:32.262391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:32.315534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972c362-0b16-4118-a422-b95ad431a86b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:32.350581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:32.351272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:32.351561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:32.351839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:32.352655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:32.353104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:32.354549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35b31-ac11-4aca-9680-86fca03df4e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:32.366747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 634/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Fri Jun 24 09:32:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:32:32.373154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:32.481973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53a29-8467-4ea1-a97f-f3ad1e6546eb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:32.523787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:32.525539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:32.525906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:32.526227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:32.527511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] RBAC: Adding query filter params (role.id=5a49d18cafb842c69d51d5c93c8b2be9, scope.project.id=52f353bca7604872a4e99359eaeef3ff, user.id=e945abf5978842a79cfaf92c479542a7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:32.527918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:32.530165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea5a04-3387-4eb5-b4ac-83163f62e971 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:32.540087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 653/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Fri Jun 24 09:32:32 2022] GET /identity/v3/role_assignments?user.id=e945abf5978842a79cfaf92c479542a7&scope.project.id=52f353bca7604872a4e99359eaeef3ff&role.id=5a49d18cafb842c69d51d5c93c8b2be9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:34.578184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448db0-61f9-4c2c-b19c-3189ee8eb0e8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:34.578779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448db0-61f9-4c2c-b19c-3189ee8eb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:34.578779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448db0-61f9-4c2c-b19c-3189ee8eb0e8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:34.580158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 635/1288] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:34.585592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:34.586009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:34.586117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:34.613828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:34.615795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:34.654215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:34.697252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 654/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:34.732311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da448db0-61f9-4c2c-b19c-3189ee8eb0e8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:34.849298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da448db0-61f9-4c2c-b19c-3189ee8eb0e8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:34.862000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:34.894268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:34.895104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:34.895394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:34.895674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:34.896334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:34.897713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:34.911442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:34.915731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:34.916784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 636/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:32:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:34.920411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:35.038652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447989c8-38d6-4454-9a27-13e5fc30e7a8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:35.078023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:35.078694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:35.078979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:35.079289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:35.080002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:35.080311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:35.081711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5da30-8efa-403c-bcd5-ce7c4a70fd93 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:35.089064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 655/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Fri Jun 24 09:32:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:35.094968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:35.216532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33527494-1523-403c-87fd-c219f1449c98 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:35.253199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:35.253881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:35.254161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:35.254434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:35.255165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:35.255472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:35.256862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b1075-1349-420a-8128-bb8e1465c73d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:35.261756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 637/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Fri Jun 24 09:32:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:37.279608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f6f7-5e49-491f-8996-dfdbb5c6a4f2 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:37.280170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f6f7-5e49-491f-8996-dfdbb5c6a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:37.280170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f6f7-5e49-491f-8996-dfdbb5c6a4f2 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:37.281542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 656/1293] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:37.287880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d26-3a83-4824-8f2a-9895af5af84a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:37.288587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d26-3a83-4824-8f2a-9895af5af84a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:37.288735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746d26-3a83-4824-8f2a-9895af5af84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:37.307794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c746d26-3a83-4824-8f2a-9895af5af84a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:37.308995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c746d26-3a83-4824-8f2a-9895af5af84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:37.341785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:37.389675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 638/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:37.416341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0f6f7-5e49-491f-8996-dfdbb5c6a4f2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:37.450867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:37.541982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0f6f7-5e49-491f-8996-dfdbb5c6a4f2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:37.577600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:37.578265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:37.578544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:37.578846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:37.579594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:37.580975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a420dd-b091-4b66-802f-cb6adf23d677 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:37.595165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 657/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Fri Jun 24 09:32:37 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:32:39.691730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519444-b5b7-410b-b7b1-ef382d0444bb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:39.692627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519444-b5b7-410b-b7b1-ef382d0444bb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:39.692627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20519444-b5b7-410b-b7b1-ef382d0444bb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:39.694812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 639/1296] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:39.703002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:39.703474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:39.703562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:39.721749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:39.722944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:39.801560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 658/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 09:32:39.826364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20519444-b5b7-410b-b7b1-ef382d0444bb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:39.947357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20519444-b5b7-410b-b7b1-ef382d0444bb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:39.990108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:39.990905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:39.991227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:39.991511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:39.992161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:39.993574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:40.005690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:40.006807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:40.007840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:40.011242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 640/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:32:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 09:32:40.013771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:40.041498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:40.134701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8fda-209c-4052-9e73-e2b8ad9e61d8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:40.169840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:40.170508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:40.170792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:40.171069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:40.171807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:40.172124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:40.173522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:40.178812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 659/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Fri Jun 24 09:32:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:40.188355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:40.223449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:40.302750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd506e08-8cc3-4e61-bef0-0b10a4486ecd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:40.347931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:40.348592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:40.348875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:40.349150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:40.349936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] RBAC: Adding query filter params (interface=public, service_id=f787f90f3eb64830b583b8e30d7f9398, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:40.350246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:40.351675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f17611-8207-4396-8eec-73e09374cb6f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:40.357599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 641/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Fri Jun 24 09:32:40 2022] GET /identity/v3/endpoints?service_id=f787f90f3eb64830b583b8e30d7f9398&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:40.363893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:40.452494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:40.472767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbabaa-3f0f-4587-a995-30b1e1e9da4a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:40.508670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:40.509200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:40.509483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:40.509756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:40.510444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:40.511857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf6320-c988-493f-acbd-273c326df228 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:40.518826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 660/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Fri Jun 24 09:32:40 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:32:42.544029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799bb4-dbad-4df8-bdb3-8bdfe35418c8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:42.544625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799bb4-dbad-4df8-bdb3-8bdfe35418c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:42.544625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799bb4-dbad-4df8-bdb3-8bdfe35418c8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:42.545976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 642/1302] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:32:42.552295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:42.553021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:42.553188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:42.573359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:32:42.574552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:32:42.649095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 661/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:32:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 09:32:42.672157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d799bb4-dbad-4df8-bdb3-8bdfe35418c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:42.777160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:42.785190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d799bb4-dbad-4df8-bdb3-8bdfe35418c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:42.821072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:42.821819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:42.822111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:42.822391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:42.823055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:42.824476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 09:32:42.836884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 09:32:42.838027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 24 09:32:42.839613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 643/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Fri Jun 24 09:32:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 09:32:42.847064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:42.929007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:42.952822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0707-471b-472b-95a7-f5fcf1b399c2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:42.986423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:42.987466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:42.987744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:42.988019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:42.988724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:32:42.989030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:42.990421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def5355-39f2-4a23-bd77-c9df444b2e60 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:42.996064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 662/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Fri Jun 24 09:32:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:32:43.002582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:32:43.111346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effb43c-4963-4b6a-aa49-4503dbe27dfc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:32:43.146387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:32:43.147033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:32:43.147354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:32:43.147629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:32:43.148363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:32:43.149764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e392c-aeda-4b55-94f4-61c7c21dd169 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:32:43.175863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 644/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Fri Jun 24 09:32:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:32:52.992927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=30 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:32:53.168489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:47.232493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db26c-ea22-43fd-bc86-f63137e30dcd None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:47.233439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db26c-ea22-43fd-bc86-f63137e30dcd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:47.233439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db26c-ea22-43fd-bc86-f63137e30dcd None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:47.238535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 663/1307] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:33:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:47.240313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6741bb-c33e-4878-a0fc-d54eb5a507a1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:47.241232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6741bb-c33e-4878-a0fc-d54eb5a507a1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:47.241746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6741bb-c33e-4878-a0fc-d54eb5a507a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:47.332049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9d6741bb-c33e-4878-a0fc-d54eb5a507a1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:47.333804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d6741bb-c33e-4878-a0fc-d54eb5a507a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:47.471097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 645/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:33:47.514211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:47.514507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:47.514774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:47.516597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 664/1309] 10.222.0.65 () {54 vars in 964 bytes} [Fri Jun 24 09:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:47.536287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd446-7858-4577-b659-16ca6863002c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:47.537536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd446-7858-4577-b659-16ca6863002c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:47.538858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd446-7858-4577-b659-16ca6863002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:47.688604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-31fcd446-7858-4577-b659-16ca6863002c None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:47.690160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31fcd446-7858-4577-b659-16ca6863002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:47.730714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:47.806010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 646/1310] 10.222.0.65 () {58 vars in 1061 bytes} [Fri Jun 24 09:33:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:33:47.815731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:33:47.817700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:47.932503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc784d3-ee24-430b-b446-4caa0e877450 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:33:48.009156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:33:48.009745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:48.010031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:48.010306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:48.011994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:48.070294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:48.133441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:33:48.134922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794fb432-4719-4adf-bff6-8df6193353c6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:33:48.157030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 665/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:33:50.472034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:50.472621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:50.472621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:50.474248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 647/1312] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:50.480002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44186d-9d1c-4697-b376-1569d495986a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:50.480493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44186d-9d1c-4697-b376-1569d495986a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:50.480570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44186d-9d1c-4697-b376-1569d495986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:50.561600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cf44186d-9d1c-4697-b376-1569d495986a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:50.562839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cf44186d-9d1c-4697-b376-1569d495986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:50.664595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 666/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:33:50.703811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:33:50.705382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:50.838962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa145844-7590-4d2e-9104-83199d791b76 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:33:50.920127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:50.922914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:33:50.923782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:50.924114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:50.924586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:50.926498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:51.043655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:33:51.045033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07140515-6708-45d0-8dbf-560af2af3aa8 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:33:51.063908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 648/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:33:53.391877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2923ed-76a0-49ee-8e69-10b68e03bac7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:53.392753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2923ed-76a0-49ee-8e69-10b68e03bac7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:53.392753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2923ed-76a0-49ee-8e69-10b68e03bac7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:53.394940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 667/1315] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:33:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:53.400339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748fff-f0da-48cb-a189-456eac0cbc56 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:53.400759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748fff-f0da-48cb-a189-456eac0cbc56 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:53.400869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748fff-f0da-48cb-a189-456eac0cbc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:53.466308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-96748fff-f0da-48cb-a189-456eac0cbc56 None None] MFA Rules not processed for user `1de5aad186984d4aae5c980eb27c005a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:53.467560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96748fff-f0da-48cb-a189-456eac0cbc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:53.520961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:53.569844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 649/1316] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:33:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:33:53.625022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:53.625280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:53.625724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:53.627992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 668/1317] 10.222.0.65 () {54 vars in 964 bytes} [Fri Jun 24 09:33:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:53.636865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa875-74c1-48ad-bf83-553ed6d85c63 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:53.637153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa875-74c1-48ad-bf83-553ed6d85c63 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:53.637436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa875-74c1-48ad-bf83-553ed6d85c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:53.715868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c15fa875-74c1-48ad-bf83-553ed6d85c63 None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:53.717092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c15fa875-74c1-48ad-bf83-553ed6d85c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:53.798831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 650/1318] 10.222.0.65 () {58 vars in 1061 bytes} [Fri Jun 24 09:33:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:33:53.808541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:33:53.810104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:53.820466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:53.920827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52548250-2449-4fb2-aebb-4d2ecb5ce78b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:33:53.997628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:33:53.998191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:53.998486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:53.998756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:54.000431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:54.065725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:54.117284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:33:54.118799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090246bf-881d-45a4-93f4-1a87e5d6db93 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:33:54.138371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 669/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:33:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:33:57.123812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:57.124396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:57.124396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:57.125637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 651/1320] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:33:57.132779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15dee1-6a43-4695-8c5d-7b7bd905928f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:57.133555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15dee1-6a43-4695-8c5d-7b7bd905928f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:57.133712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15dee1-6a43-4695-8c5d-7b7bd905928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:57.202844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ea15dee1-6a43-4695-8c5d-7b7bd905928f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:33:57.204088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ea15dee1-6a43-4695-8c5d-7b7bd905928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:57.302717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 670/1321] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:33:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:33:57.345428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:33:57.347974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:57.404978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:57.473489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a641c0-9233-4ce6-ab21-38489be80302 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:33:57.552857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:33:57.553517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:33:57.553800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:33:57.554083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:33:57.555775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:33:57.657660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:33:57.672312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:33:57.673757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f3617-fdfa-47e5-8402-bee3d7242093 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:33:57.695222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 652/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:33:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:00.469242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:00.469824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:00.469824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:00.471391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 671/1323] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:34:00.476646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89fa5-8607-4e31-a991-07d380ff4a7c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:00.477098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89fa5-8607-4e31-a991-07d380ff4a7c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:00.477202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89fa5-8607-4e31-a991-07d380ff4a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:00.547217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ab89fa5-8607-4e31-a991-07d380ff4a7c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:00.548445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ab89fa5-8607-4e31-a991-07d380ff4a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:00.656240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 653/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:00.693392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:00.695040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:00.716584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:00.816718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee27ac-0872-434e-bdbf-d11c7fe58387 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:00.894694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:00.895443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:00.895721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:00.895997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:00.897631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:00.968667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:01.009179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:01.010530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b3c1-5600-4dfa-aefa-2d35d6bf93c6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:01.030763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 672/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:03.136823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:03.137338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:03.137338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:03.138786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 654/1326] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:03.144188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2c6e8-00e6-4170-a74c-7527adf485f9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:03.144615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2c6e8-00e6-4170-a74c-7527adf485f9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:03.145098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2c6e8-00e6-4170-a74c-7527adf485f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:03.210883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-76b2c6e8-00e6-4170-a74c-7527adf485f9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:03.212098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-76b2c6e8-00e6-4170-a74c-7527adf485f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:03.309883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 673/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:03.345976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:03.347672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:03.410693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:03.456743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b1b14-8c0b-431d-b209-857ad17b8c40 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:03.539899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:03.540564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:03.540849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:03.541135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:03.542778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:03.659377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:03.662273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:03.663924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe98ed-2305-4720-af61-3609a1ba3150 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:03.685243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 655/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:05.801887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:05.802456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:05.802456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:05.803859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 674/1329] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:05.809465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f7012-bcd4-4948-b80f-e96ddc69447d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:05.809916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f7012-bcd4-4948-b80f-e96ddc69447d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:05.810094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f7012-bcd4-4948-b80f-e96ddc69447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:05.894977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c8f7012-bcd4-4948-b80f-e96ddc69447d None None] MFA Rules not processed for user `1de5aad186984d4aae5c980eb27c005a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:05.896228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c8f7012-bcd4-4948-b80f-e96ddc69447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:05.999867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 656/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:06.037114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:06.038769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:06.070845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:06.142537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34f71d-6fd8-43c0-9cea-79569d3bb14f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:06.220855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:06.221969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:06.221969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:06.222511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:06.226337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:06.312655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:06.349030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:06.350502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcaba9-5404-4e5d-9895-ae4a7d1159e6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:06.370892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 675/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:08.678048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:08.678648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:08.678648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:08.680015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 657/1332] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:08.686376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79225e80-9a28-487c-980a-c48f0d892c9e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:08.687171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79225e80-9a28-487c-980a-c48f0d892c9e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:08.687378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79225e80-9a28-487c-980a-c48f0d892c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:08.766926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-79225e80-9a28-487c-980a-c48f0d892c9e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:08.768212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-79225e80-9a28-487c-980a-c48f0d892c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:08.873741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 676/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:08.917273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:08.920383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:08.994626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:09.032567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ee9fa-44ef-4427-be59-1d55bacd4764 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:09.137771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:09.138435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:09.138726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:09.138998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:09.140686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:09.265357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:09.266718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3275c-cd7e-431c-8064-1e0818040e65 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:09.277499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:09.288937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 658/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:34:11.454257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:11.454856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:11.454856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:11.456972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 677/1335] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:11.462684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf695e-8cbe-4975-a072-362da3b3d777 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:11.463220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf695e-8cbe-4975-a072-362da3b3d777 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:11.463220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf695e-8cbe-4975-a072-362da3b3d777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:11.539969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-28bf695e-8cbe-4975-a072-362da3b3d777 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:11.541215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-28bf695e-8cbe-4975-a072-362da3b3d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:11.663481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 659/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:11.729033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:11.730578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:11.778750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:11.845610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065ff4-2869-4b7a-8057-6b237f64c818 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:11.927961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:11.928626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:11.928934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:11.929211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:11.930946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:12.022030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:12.049142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:12.050782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa8e9d-23c0-49df-8bcf-096d1dd34843 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:12.069885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 678/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:14.588755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:14.589288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:14.589288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:14.590605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 660/1338] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:14.596918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2462f7-4589-460f-b66f-539cc90398eb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:14.597812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2462f7-4589-460f-b66f-539cc90398eb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:14.598093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2462f7-4589-460f-b66f-539cc90398eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:14.678068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-af2462f7-4589-460f-b66f-539cc90398eb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:14.679321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-af2462f7-4589-460f-b66f-539cc90398eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:14.783556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 679/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:14.819960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:14.821518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:14.901266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:14.931190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299074d7-7027-439d-9c7e-9d1142a5eb28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:15.016365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:15.017039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:15.017322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:15.017588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:15.019912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:15.137662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:15.139111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d368d4f-c58a-40c0-b7c3-9449c8b269b2 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:15.159118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 661/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:19.355538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:19.356421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:19.356421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:19.358092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 680/1341] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:19.363466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19194-1719-4e76-9a9e-642d3446900f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:19.363937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19194-1719-4e76-9a9e-642d3446900f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:19.364041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19194-1719-4e76-9a9e-642d3446900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:19.371345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:19.440543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09b19194-1719-4e76-9a9e-642d3446900f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:19.441809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09b19194-1719-4e76-9a9e-642d3446900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:19.550165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 662/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:34:19.588481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:19.590982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:19.643419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:19.691789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f975a6-0a88-4562-ab22-72242d4d939b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:19.695429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:19.696075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:19.696352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:19.696627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:19.705863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:19.707446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f2f5a-7c55-42f1-afef-33be9b085781 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:19.717571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 681/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:22.343335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34f3c-87de-41b9-9fa4-24256420dc3d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:22.343946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34f3c-87de-41b9-9fa4-24256420dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:22.343946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e34f3c-87de-41b9-9fa4-24256420dc3d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:22.345253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 663/1344] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:22.350602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d07cc-c15e-4a87-b1e7-bb6134385ef1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:22.351037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d07cc-c15e-4a87-b1e7-bb6134385ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:22.351178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d07cc-c15e-4a87-b1e7-bb6134385ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:22.380124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-623d07cc-c15e-4a87-b1e7-bb6134385ef1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:22.381296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-623d07cc-c15e-4a87-b1e7-bb6134385ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:22.462989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 682/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:22.503625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e34f3c-87de-41b9-9fa4-24256420dc3d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:22.520373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e34f3c-87de-41b9-9fa4-24256420dc3d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:22.526171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:22.526881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:22.527218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:22.527490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:22.544808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:22.546362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b3241-aebe-4e2f-8d90-c1879ac4c1f5 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:22.557661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 664/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:26.917013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2298ff-68cb-47d7-82b6-470afc6498b8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:26.917641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2298ff-68cb-47d7-82b6-470afc6498b8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:26.917641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2298ff-68cb-47d7-82b6-470afc6498b8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:26.919386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 683/1347] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:26.925480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c0ee-3992-46bd-af89-28122eafda52 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:26.925954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c0ee-3992-46bd-af89-28122eafda52 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:26.926189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98c0ee-3992-46bd-af89-28122eafda52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:26.945086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b98c0ee-3992-46bd-af89-28122eafda52 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:26.946290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b98c0ee-3992-46bd-af89-28122eafda52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:26.954497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 665/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:26.993557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2298ff-68cb-47d7-82b6-470afc6498b8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:27.006563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2298ff-68cb-47d7-82b6-470afc6498b8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:27.010457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:27.011113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:27.011426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:27.011705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:27.021114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:27.022600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97893cc-b837-433c-9d6f-1ac1d62401dc service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:27.032127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 684/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:32.016460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630d43-5f47-42de-8a14-4aed2f5ea3e0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:32.016952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630d43-5f47-42de-8a14-4aed2f5ea3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:32.017058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83630d43-5f47-42de-8a14-4aed2f5ea3e0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:32.019426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 666/1350] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:32.025532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463bf5-17c8-4e2a-9745-57b4d823082c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:32.025972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463bf5-17c8-4e2a-9745-57b4d823082c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:32.026090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463bf5-17c8-4e2a-9745-57b4d823082c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:32.119034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cd463bf5-17c8-4e2a-9745-57b4d823082c None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:32.126687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cd463bf5-17c8-4e2a-9745-57b4d823082c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:32.163792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:32.240861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 685/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:34:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:36.585010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3cd81-aee9-4632-a5ff-c3a0761e259e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:36.585578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3cd81-aee9-4632-a5ff-c3a0761e259e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:36.585578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3cd81-aee9-4632-a5ff-c3a0761e259e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:36.587123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 667/1352] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:36.592106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5f59e-9af3-4f71-be0f-c07f432108f4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:36.592533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5f59e-9af3-4f71-be0f-c07f432108f4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:36.592722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5f59e-9af3-4f71-be0f-c07f432108f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:36.613125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c1f5f59e-9af3-4f71-be0f-c07f432108f4 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:36.614429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c1f5f59e-9af3-4f71-be0f-c07f432108f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:36.625009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 686/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:34:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:36.939187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=72 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:38.056558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffc4be-a864-4223-b565-27bf29c732c4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:38.057193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffc4be-a864-4223-b565-27bf29c732c4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:38.057193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffc4be-a864-4223-b565-27bf29c732c4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:38.058505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 668/1354] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:38.063550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ccba-4ef6-4c5c-87ac-310e88483f16 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:38.063953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ccba-4ef6-4c5c-87ac-310e88483f16 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:38.064259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186ccba-4ef6-4c5c-87ac-310e88483f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:38.082891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9186ccba-4ef6-4c5c-87ac-310e88483f16 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:38.084151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9186ccba-4ef6-4c5c-87ac-310e88483f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:38.093401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 687/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:34:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:41.733711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65d77-0428-4ca2-924b-4762df6de223 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:41.734331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65d77-0428-4ca2-924b-4762df6de223 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:41.734331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65d77-0428-4ca2-924b-4762df6de223 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:41.736758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 669/1356] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:41.744821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9e0e-5c22-47bd-8f2d-8537f0f1f0e8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:41.744821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9e0e-5c22-47bd-8f2d-8537f0f1f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:41.744821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca9e0e-5c22-47bd-8f2d-8537f0f1f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:41.763823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c3ca9e0e-5c22-47bd-8f2d-8537f0f1f0e8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:41.766948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c3ca9e0e-5c22-47bd-8f2d-8537f0f1f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:41.776856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 688/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:41.837596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1513-befd-4b7e-9697-11ac06a76ce6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:41.837596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1513-befd-4b7e-9697-11ac06a76ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:41.837596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1513-befd-4b7e-9697-11ac06a76ce6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:41.838556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 670/1358] 10.222.0.65 () {54 vars in 961 bytes} [Fri Jun 24 09:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:41.858012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8e98-2b3c-46d3-87cf-d423e47e8bb9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:41.858671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8e98-2b3c-46d3-87cf-d423e47e8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:41.859199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8e98-2b3c-46d3-87cf-d423e47e8bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:41.911181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-989a8e98-2b3c-46d3-87cf-d423e47e8bb9 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:41.912488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-989a8e98-2b3c-46d3-87cf-d423e47e8bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:41.999227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 689/1359] 10.222.0.65 () {58 vars in 1058 bytes} [Fri Jun 24 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:42.007390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1513-befd-4b7e-9697-11ac06a76ce6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:42.021937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1513-befd-4b7e-9697-11ac06a76ce6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:42.028487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:42.029218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:42.029571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:42.029915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:42.039009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:42.040788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85192e1-ba40-4dad-a52c-996e764a039f service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:42.051320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 671/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:34:44.273137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d001a-50b5-4177-9a00-9a0dfbf0a2dd None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:44.273700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d001a-50b5-4177-9a00-9a0dfbf0a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:44.274264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d001a-50b5-4177-9a00-9a0dfbf0a2dd None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:44.276148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 690/1361] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:44.281008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad805bc-69ec-434b-b9c7-94836b561add None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:44.281486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad805bc-69ec-434b-b9c7-94836b561add None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:44.281594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad805bc-69ec-434b-b9c7-94836b561add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:44.300224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2ad805bc-69ec-434b-b9c7-94836b561add None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:44.301422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ad805bc-69ec-434b-b9c7-94836b561add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:44.309683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 672/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:44.353718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d001a-50b5-4177-9a00-9a0dfbf0a2dd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:44.369656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d001a-50b5-4177-9a00-9a0dfbf0a2dd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:44.375627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:44.376432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:44.376722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:44.377002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:44.386345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:44.388021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b622a0-7608-4a9a-b42e-538f8b21827d service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:44.398291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 691/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:46.803456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cad35-19aa-46b6-ae1b-9ea489b28c6b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:46.803917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cad35-19aa-46b6-ae1b-9ea489b28c6b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:46.803917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cad35-19aa-46b6-ae1b-9ea489b28c6b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:46.805763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 673/1364] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:46.811882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982f50-5a14-4ba9-8d4d-77922f91234d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:46.811882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982f50-5a14-4ba9-8d4d-77922f91234d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:46.812372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14982f50-5a14-4ba9-8d4d-77922f91234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:46.832729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-14982f50-5a14-4ba9-8d4d-77922f91234d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:46.833883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-14982f50-5a14-4ba9-8d4d-77922f91234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:46.843567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 692/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:46.891365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20ed7d-7052-417d-b0d1-12f3b54fd643 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:46.894119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20ed7d-7052-417d-b0d1-12f3b54fd643 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:46.894309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20ed7d-7052-417d-b0d1-12f3b54fd643 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:46.895746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 674/1366] 10.222.0.65 () {54 vars in 961 bytes} [Fri Jun 24 09:34:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:46.905751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e38dc-26a8-489e-8b7c-bad04ac9b20a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:46.906408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e38dc-26a8-489e-8b7c-bad04ac9b20a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:46.906886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e38dc-26a8-489e-8b7c-bad04ac9b20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:46.937373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-356e38dc-26a8-489e-8b7c-bad04ac9b20a None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:46.938704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-356e38dc-26a8-489e-8b7c-bad04ac9b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:46.949612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 693/1367] 10.222.0.65 () {58 vars in 1058 bytes} [Fri Jun 24 09:34:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:46.958240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20ed7d-7052-417d-b0d1-12f3b54fd643 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:46.973901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20ed7d-7052-417d-b0d1-12f3b54fd643 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:46.980293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:46.980961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:46.981248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:46.981524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:46.990600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:46.992239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997476f4-258e-4859-8924-167b1e11def8 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:47.002183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 675/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:49.117655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fc5a-9bcf-44e6-a6bc-a812239d3d61 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:49.118187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fc5a-9bcf-44e6-a6bc-a812239d3d61 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:49.118304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2fc5a-9bcf-44e6-a6bc-a812239d3d61 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:49.119858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 694/1369] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:49.125249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443abe0-7980-4866-a5dc-f42fb59e1fd1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:49.125827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443abe0-7980-4866-a5dc-f42fb59e1fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:49.125827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443abe0-7980-4866-a5dc-f42fb59e1fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:49.144865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7443abe0-7980-4866-a5dc-f42fb59e1fd1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:49.146362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7443abe0-7980-4866-a5dc-f42fb59e1fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:49.154867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 676/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:49.201549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2fc5a-9bcf-44e6-a6bc-a812239d3d61 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:49.225028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2fc5a-9bcf-44e6-a6bc-a812239d3d61 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:49.233255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:49.234313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:49.234645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:49.234923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:49.247068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:49.249283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f985d0-2098-4e63-8726-c92358427d4d service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:49.265093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 695/1371] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:49.931903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abb0b-b383-4241-a220-f5142a366842 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:49.932162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abb0b-b383-4241-a220-f5142a366842 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:49.932427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abb0b-b383-4241-a220-f5142a366842 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:49.933800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 677/1372] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:49.940179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754b8f-62f7-4069-9dbd-9c42f482b048 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:49.940499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754b8f-62f7-4069-9dbd-9c42f482b048 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:49.940783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754b8f-62f7-4069-9dbd-9c42f482b048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:49.959809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-88754b8f-62f7-4069-9dbd-9c42f482b048 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:49.961020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-88754b8f-62f7-4069-9dbd-9c42f482b048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:49.971119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 696/1373] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:50.281611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc72c9-ab24-473e-aa3b-538ead585de7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:50.282205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc72c9-ab24-473e-aa3b-538ead585de7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:50.282289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc72c9-ab24-473e-aa3b-538ead585de7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:50.284231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 678/1374] 10.222.0.65 () {54 vars in 963 bytes} [Fri Jun 24 09:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:50.289703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea408-ed98-4efa-b29c-e4b82b46d1f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:50.290263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea408-ed98-4efa-b29c-e4b82b46d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:50.290263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea408-ed98-4efa-b29c-e4b82b46d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:50.297079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:34:50.342933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-751ea408-ed98-4efa-b29c-e4b82b46d1f6 None None] MFA Rules not processed for user `fd8aa4f4a5cb441cae26c8dafa27548c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:50.344167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-751ea408-ed98-4efa-b29c-e4b82b46d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:50.406723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 697/1375] 10.222.0.65 () {58 vars in 1060 bytes} [Fri Jun 24 09:34:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:50.414636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc72c9-ab24-473e-aa3b-538ead585de7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:50.426735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc72c9-ab24-473e-aa3b-538ead585de7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:50.431462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:50.432415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:50.432925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:50.433384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:50.444816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:50.446763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8c334-0d47-4d05-a0df-79a952c3dc7a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:50.458616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 679/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:51.363622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae532c-2f7c-4bb4-b749-e75009d32177 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:51.364204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae532c-2f7c-4bb4-b749-e75009d32177 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:51.364204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae532c-2f7c-4bb4-b749-e75009d32177 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:51.366526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 698/1377] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:51.371389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c26486-98bd-44fb-8ab6-c0fffc82cf74 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:51.372181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c26486-98bd-44fb-8ab6-c0fffc82cf74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:51.372181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c26486-98bd-44fb-8ab6-c0fffc82cf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:51.390220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-86c26486-98bd-44fb-8ab6-c0fffc82cf74 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:51.392266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86c26486-98bd-44fb-8ab6-c0fffc82cf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:51.403032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 680/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:34:51.448241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae532c-2f7c-4bb4-b749-e75009d32177 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:51.464887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae532c-2f7c-4bb4-b749-e75009d32177 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:51.470790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:51.471698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:51.472077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:51.472415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:51.481475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:51.483110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861e10d-b739-4139-a438-a8222e1e3a12 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:51.493199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 699/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:53.550578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3127-7b0f-4c01-9bc8-774e3a796771 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:53.551168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3127-7b0f-4c01-9bc8-774e3a796771 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:53.551168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3127-7b0f-4c01-9bc8-774e3a796771 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:53.553297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 681/1380] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:53.558979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4450f-7517-4b01-adca-1fce8052ebb6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:53.559605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4450f-7517-4b01-adca-1fce8052ebb6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:53.559899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4450f-7517-4b01-adca-1fce8052ebb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:53.592418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-43e4450f-7517-4b01-adca-1fce8052ebb6 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:53.593755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43e4450f-7517-4b01-adca-1fce8052ebb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:53.606213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 700/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:34:53.650114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3127-7b0f-4c01-9bc8-774e3a796771 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:53.665080 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3127-7b0f-4c01-9bc8-774e3a796771 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:53.670881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:53.671579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:53.671858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:53.672134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:53.680950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:53.682539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2199a14-309e-4748-b921-29358fd262f5 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:53.692052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 682/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:55.779233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e0c62-42ba-458a-ba72-c6796d38fc31 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:55.780142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e0c62-42ba-458a-ba72-c6796d38fc31 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:55.780515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e0c62-42ba-458a-ba72-c6796d38fc31 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:55.782777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 701/1383] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:55.788454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4922d4-cc6b-4b8e-b9a7-83eb2b31ae7f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:55.788935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4922d4-cc6b-4b8e-b9a7-83eb2b31ae7f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:55.789198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4922d4-cc6b-4b8e-b9a7-83eb2b31ae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:55.807912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e4922d4-cc6b-4b8e-b9a7-83eb2b31ae7f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:55.809099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e4922d4-cc6b-4b8e-b9a7-83eb2b31ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:55.817515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 683/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:34:55.861522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e0c62-42ba-458a-ba72-c6796d38fc31 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:55.877107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e0c62-42ba-458a-ba72-c6796d38fc31 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:55.882992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:55.883786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:55.884214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:55.884579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:55.894642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:55.896393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ca7b4-1d51-4c4b-85fc-9795a32705ff service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:55.906536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 702/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:34:57.957889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f310-b5d7-4889-944b-aae132e13531 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:57.959230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f310-b5d7-4889-944b-aae132e13531 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:57.959230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86f310-b5d7-4889-944b-aae132e13531 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:57.963581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 684/1386] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:34:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:34:57.964907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26057f49-d5c5-4aed-8c98-70c9990fd275 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:57.964907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26057f49-d5c5-4aed-8c98-70c9990fd275 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:57.966267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26057f49-d5c5-4aed-8c98-70c9990fd275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:57.983972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-26057f49-d5c5-4aed-8c98-70c9990fd275 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:34:57.985137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-26057f49-d5c5-4aed-8c98-70c9990fd275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:34:58.007459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 703/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:34:58.038623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f310-b5d7-4889-944b-aae132e13531 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:34:58.054846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86f310-b5d7-4889-944b-aae132e13531 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:34:58.060181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:34:58.060935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:34:58.061189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:34:58.061365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:34:58.070554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:34:58.072222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e252-4358-40f6-a995-26d2cdad3aa1 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:34:58.082258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 685/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:00.248607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae54c-473b-45f7-86e6-516e28f8c75d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:00.249194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae54c-473b-45f7-86e6-516e28f8c75d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:00.249194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae54c-473b-45f7-86e6-516e28f8c75d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:00.251113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 704/1389] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:00.257755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb55af-09c7-4e6a-97cd-93215f1de327 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:00.258396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb55af-09c7-4e6a-97cd-93215f1de327 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:00.258396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb55af-09c7-4e6a-97cd-93215f1de327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:00.279333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f9eb55af-09c7-4e6a-97cd-93215f1de327 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:00.280963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f9eb55af-09c7-4e6a-97cd-93215f1de327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:00.291358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 686/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:00.337884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae54c-473b-45f7-86e6-516e28f8c75d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:00.358226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae54c-473b-45f7-86e6-516e28f8c75d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:00.366312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:00.367186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:00.367837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:00.368204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:00.380126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:00.382396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833fcb7d-0962-4694-811d-b7319cb754d6 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:00.397259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 705/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:02.756258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc28cc-aa2e-4baa-9474-86bd196b23b6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:02.756921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc28cc-aa2e-4baa-9474-86bd196b23b6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:02.756921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc28cc-aa2e-4baa-9474-86bd196b23b6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:02.758261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 687/1392] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:02.765413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d131-8e63-4ded-8d0c-345d33728789 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:02.766715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d131-8e63-4ded-8d0c-345d33728789 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:02.766715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40d131-8e63-4ded-8d0c-345d33728789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:02.789543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8d40d131-8e63-4ded-8d0c-345d33728789 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:02.791448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8d40d131-8e63-4ded-8d0c-345d33728789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:02.803173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 706/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:02.861650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc28cc-aa2e-4baa-9474-86bd196b23b6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:02.889284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc28cc-aa2e-4baa-9474-86bd196b23b6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:02.896618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:02.897414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:02.897758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:02.898038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:02.907845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:02.909476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc22a-a892-4086-b2d3-b5c6414ca23a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:02.919587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 688/1394] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:05.058822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a331e-baa8-4deb-8334-84313c1e37ea None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.059392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a331e-baa8-4deb-8334-84313c1e37ea None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.059392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a331e-baa8-4deb-8334-84313c1e37ea None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.060715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 707/1395] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:05.066282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d474e-9f41-4d6a-b8db-817c835b0ac0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.066741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d474e-9f41-4d6a-b8db-817c835b0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.066845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d474e-9f41-4d6a-b8db-817c835b0ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.085056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-db9d474e-9f41-4d6a-b8db-817c835b0ac0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:05.086229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-db9d474e-9f41-4d6a-b8db-817c835b0ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:05.094676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 689/1396] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:05.139741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a331e-baa8-4deb-8334-84313c1e37ea None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:05.155451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a331e-baa8-4deb-8334-84313c1e37ea None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:05.161172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:05.161830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.162109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.162379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.171246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:05.172978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37214065-f99b-4f50-853a-79a05a481c36 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:05.182768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 708/1397] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:05.711951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58820-9103-456e-b83b-b26f5d0b49b5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.712236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58820-9103-456e-b83b-b26f5d0b49b5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.712512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58820-9103-456e-b83b-b26f5d0b49b5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.713916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 690/1398] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:05.719356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab502b1-7d90-4f64-84af-6ce8162e7bb3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.719645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab502b1-7d90-4f64-84af-6ce8162e7bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.719920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab502b1-7d90-4f64-84af-6ce8162e7bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.738772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1ab502b1-7d90-4f64-84af-6ce8162e7bb3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:05.739993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1ab502b1-7d90-4f64-84af-6ce8162e7bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:05.748334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 709/1399] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:05.803566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62471d40-4788-464a-bca3-94bc97ae8d26 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.803855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62471d40-4788-464a-bca3-94bc97ae8d26 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.804141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62471d40-4788-464a-bca3-94bc97ae8d26 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.805501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 691/1400] 10.222.0.65 () {54 vars in 962 bytes} [Fri Jun 24 09:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:05.810658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45019-3a77-42ed-bb2c-2bce322e3aff None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.810953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45019-3a77-42ed-bb2c-2bce322e3aff None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.811270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45019-3a77-42ed-bb2c-2bce322e3aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.863030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-00f45019-3a77-42ed-bb2c-2bce322e3aff None None] MFA Rules not processed for user `01aa5644b3834b0d887bbfc854ebc8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:05.864413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-00f45019-3a77-42ed-bb2c-2bce322e3aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:05.897346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:05.926487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 710/1401] 10.222.0.65 () {58 vars in 1059 bytes} [Fri Jun 24 09:35:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:05.934221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62471d40-4788-464a-bca3-94bc97ae8d26 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:05.946236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62471d40-4788-464a-bca3-94bc97ae8d26 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:05.950708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:05.951403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:05.951689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:05.951966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:05.962895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:05.964310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b50bcc-6935-4ec8-8929-b11c444944e2 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:05.974547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 692/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:06.697265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c2df-ae01-424f-8fc1-4f8721cefc4c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:06.697745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c2df-ae01-424f-8fc1-4f8721cefc4c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:06.697862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7c2df-ae01-424f-8fc1-4f8721cefc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:06.749079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-30b7c2df-ae01-424f-8fc1-4f8721cefc4c None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:06.750483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-30b7c2df-ae01-424f-8fc1-4f8721cefc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:06.816631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 711/1403] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:35:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:06.858750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2b47c-f93f-4f54-8e15-5d3cc2180a14 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:06.859436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2b47c-f93f-4f54-8e15-5d3cc2180a14 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:06.859730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2b47c-f93f-4f54-8e15-5d3cc2180a14 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:06.861168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 693/1404] 10.222.0.65 () {54 vars in 942 bytes} [Fri Jun 24 09:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:06.867843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df298a-4c8d-4a20-8926-9e5528971922 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:06.868160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df298a-4c8d-4a20-8926-9e5528971922 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:06.868489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df298a-4c8d-4a20-8926-9e5528971922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:06.918080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-29df298a-4c8d-4a20-8926-9e5528971922 None None] MFA Rules not processed for user `616b28448ac548c0b9c22f1245b090b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:06.919549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-29df298a-4c8d-4a20-8926-9e5528971922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:06.997166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 712/1405] 10.222.0.65 () {58 vars in 1039 bytes} [Fri Jun 24 09:35:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:07.007260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2b47c-f93f-4f54-8e15-5d3cc2180a14 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:07.020787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=61 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:07.028382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2b47c-f93f-4f54-8e15-5d3cc2180a14 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:07.034197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:07.034848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:07.035156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:07.035434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:07.043922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:07.045547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93730a4e-14bf-40a5-a97b-7436748f4ce3 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:07.054522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 694/1406] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:35:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:35:07.241935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acbd1-7bf0-4ccb-83ed-c8f8c1a89e2c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:07.242224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acbd1-7bf0-4ccb-83ed-c8f8c1a89e2c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:07.242504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acbd1-7bf0-4ccb-83ed-c8f8c1a89e2c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:07.243936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 713/1407] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:07.249252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707163b-b0b0-49b4-bc53-e1125d8486d8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:07.249758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707163b-b0b0-49b4-bc53-e1125d8486d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:07.250206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707163b-b0b0-49b4-bc53-e1125d8486d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:07.268989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b707163b-b0b0-49b4-bc53-e1125d8486d8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:07.270439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b707163b-b0b0-49b4-bc53-e1125d8486d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:07.283187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 695/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:07.334087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11acbd1-7bf0-4ccb-83ed-c8f8c1a89e2c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:07.348912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11acbd1-7bf0-4ccb-83ed-c8f8c1a89e2c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:07.355009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:07.355803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:07.356158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:07.356494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:07.367366 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:07.369070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da135b-a25a-4e80-b858-9a642404531c service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:07.374808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:07.386672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 714/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:17.263560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:29.356298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c404d-069d-49a4-9beb-e64d72c76e99 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:29.357018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c404d-069d-49a4-9beb-e64d72c76e99 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:29.357018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c404d-069d-49a4-9beb-e64d72c76e99 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:29.358817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 696/1410] 10.222.0.65 () {54 vars in 918 bytes} [Fri Jun 24 09:35:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:29.365883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776306a-fa44-4643-aa95-571686f39509 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:29.366389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776306a-fa44-4643-aa95-571686f39509 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:29.366389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776306a-fa44-4643-aa95-571686f39509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:29.385946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a776306a-fa44-4643-aa95-571686f39509 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:29.387132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a776306a-fa44-4643-aa95-571686f39509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:29.396662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 715/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Fri Jun 24 09:35:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:29.411483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01432a7-ac3e-4782-a1bc-3f6f583659e2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:29.411770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01432a7-ac3e-4782-a1bc-3f6f583659e2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:29.412053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01432a7-ac3e-4782-a1bc-3f6f583659e2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:29.413503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 697/1412] 10.222.0.65 () {54 vars in 918 bytes} [Fri Jun 24 09:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:29.418020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7d07-b7d7-4d3d-ad52-8bc378a03435 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:29.418306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7d07-b7d7-4d3d-ad52-8bc378a03435 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:29.418579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e7d07-b7d7-4d3d-ad52-8bc378a03435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:29.437241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0f8e7d07-b7d7-4d3d-ad52-8bc378a03435 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:29.438971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0f8e7d07-b7d7-4d3d-ad52-8bc378a03435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:29.448801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 716/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Fri Jun 24 09:35:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:31.995809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584f76-95ff-4e4f-9ac9-567212cd70b9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:31.996348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584f76-95ff-4e4f-9ac9-567212cd70b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:31.996348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584f76-95ff-4e4f-9ac9-567212cd70b9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:31.997629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:32.007501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe8a1-2ec4-41cb-9d3e-f04a5c45ba65 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:32.008252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe8a1-2ec4-41cb-9d3e-f04a5c45ba65 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:32.008423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbe8a1-2ec4-41cb-9d3e-f04a5c45ba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:32.030575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-55fbe8a1-2ec4-41cb-9d3e-f04a5c45ba65 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:32.032152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55fbe8a1-2ec4-41cb-9d3e-f04a5c45ba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:32.041527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:32.089338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57584f76-95ff-4e4f-9ac9-567212cd70b9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:32.110954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57584f76-95ff-4e4f-9ac9-567212cd70b9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:32.116669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:32.117320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:32.117607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:32.117886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:32.126204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:32.127856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc93fe-8ca8-4bd7-bff8-9a63534940cc service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:32.137313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 699/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:33.064766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369aa4b-d4d0-4244-93bb-18e101a6ce9d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:33.065642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369aa4b-d4d0-4244-93bb-18e101a6ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:33.065642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369aa4b-d4d0-4244-93bb-18e101a6ce9d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:33.067769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:33.072858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327813-da3e-46d2-a15b-d4fd1c4f6d14 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:33.073347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327813-da3e-46d2-a15b-d4fd1c4f6d14 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:33.073452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327813-da3e-46d2-a15b-d4fd1c4f6d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:33.093586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-14327813-da3e-46d2-a15b-d4fd1c4f6d14 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:33.094829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-14327813-da3e-46d2-a15b-d4fd1c4f6d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:33.103062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:33.251942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369aa4b-d4d0-4244-93bb-18e101a6ce9d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:33.273988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369aa4b-d4d0-4244-93bb-18e101a6ce9d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:33.280787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:33.281431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:33.281775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:33.282053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:33.290550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:33.292175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe3792-dd80-48b1-95c1-994ef400483a service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:33.301943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 719/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:34.202870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc446a1-1545-43b0-8d5c-00ff897c7305 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:34.203576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc446a1-1545-43b0-8d5c-00ff897c7305 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:34.203576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc446a1-1545-43b0-8d5c-00ff897c7305 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:34.204943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:34.211321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e415-4077-4310-ac82-d902c5d9d0ad None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:34.212108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e415-4077-4310-ac82-d902c5d9d0ad None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:34.212251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e415-4077-4310-ac82-d902c5d9d0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:34.241339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0536e415-4077-4310-ac82-d902c5d9d0ad None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:34.242580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0536e415-4077-4310-ac82-d902c5d9d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:34.251361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:35:34.410244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc446a1-1545-43b0-8d5c-00ff897c7305 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:34.426756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc446a1-1545-43b0-8d5c-00ff897c7305 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:34.433265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:34.433954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:34.434245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:34.434520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:34.446146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:34.448768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e907a25-c2dc-4835-9251-a4f0593722b6 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:34.464375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 702/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:35:35.357559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c2b6-3daa-43b3-9daa-57fb16cf8790 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:35.358486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c2b6-3daa-43b3-9daa-57fb16cf8790 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:35.358486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6c2b6-3daa-43b3-9daa-57fb16cf8790 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:35.361298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:35.369584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912a99c-d5e2-4aee-9dad-1eadc2358d78 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:35.370013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912a99c-d5e2-4aee-9dad-1eadc2358d78 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:35.370200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912a99c-d5e2-4aee-9dad-1eadc2358d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:35.390996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4912a99c-d5e2-4aee-9dad-1eadc2358d78 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:35.392282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4912a99c-d5e2-4aee-9dad-1eadc2358d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:35.401203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:35.566645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c2b6-3daa-43b3-9daa-57fb16cf8790 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:35.587328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6c2b6-3daa-43b3-9daa-57fb16cf8790 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:35.593434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:35.594105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:35.594668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:35.594668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:35.602961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:35.604963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa02a0-abfd-4fa0-93e4-537729a8ea89 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:35.614574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 722/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:36.481635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fed25-b42d-4961-bc09-d3d9318559bf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:36.482186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fed25-b42d-4961-bc09-d3d9318559bf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:36.482186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fed25-b42d-4961-bc09-d3d9318559bf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:36.483680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:36.489922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1966-1ebf-4717-b477-c5dc3b4aa97d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:36.490347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1966-1ebf-4717-b477-c5dc3b4aa97d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:36.490515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1966-1ebf-4717-b477-c5dc3b4aa97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:36.519533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-06ca1966-1ebf-4717-b477-c5dc3b4aa97d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:36.520761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-06ca1966-1ebf-4717-b477-c5dc3b4aa97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:36.530290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:36.680735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fed25-b42d-4961-bc09-d3d9318559bf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:36.700157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fed25-b42d-4961-bc09-d3d9318559bf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:36.706673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:36.707398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:36.707717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:36.707995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:36.716429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:36.718036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66fbf6-1e4d-48eb-b034-f914e499e510 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:36.728038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 705/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:37.555780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d779a-879b-4cce-bfe0-f5de7847fa68 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:37.556735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d779a-879b-4cce-bfe0-f5de7847fa68 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:37.556735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d779a-879b-4cce-bfe0-f5de7847fa68 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:37.558777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:37.566934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13fb8-ab55-4112-a11d-1e62a73d60f5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:37.567410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13fb8-ab55-4112-a11d-1e62a73d60f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:37.567556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13fb8-ab55-4112-a11d-1e62a73d60f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:37.595721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-20d13fb8-ab55-4112-a11d-1e62a73d60f5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:37.596935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-20d13fb8-ab55-4112-a11d-1e62a73d60f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:37.605985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:37.756605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d779a-879b-4cce-bfe0-f5de7847fa68 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:37.773479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d779a-879b-4cce-bfe0-f5de7847fa68 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:37.780180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:37.780838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:37.781126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:37.781409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:37.789697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:37.791335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926aed7-c3fe-401e-b611-70e6aa63fcac service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:37.800821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 725/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:38.625015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c7f6-0096-425c-aeab-7ca65de46eb1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:38.626014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c7f6-0096-425c-aeab-7ca65de46eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:38.626014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c7f6-0096-425c-aeab-7ca65de46eb1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:38.627718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:38.635244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1c3d1-32f4-4841-9129-6db218b720b1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:38.635960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1c3d1-32f4-4841-9129-6db218b720b1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:38.636157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1c3d1-32f4-4841-9129-6db218b720b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:38.656341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a4e1c3d1-32f4-4841-9129-6db218b720b1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:38.657672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a4e1c3d1-32f4-4841-9129-6db218b720b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:38.666689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:38.817814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c7f6-0096-425c-aeab-7ca65de46eb1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:38.834706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c7f6-0096-425c-aeab-7ca65de46eb1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:38.841528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:38.842189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:38.842473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:38.842759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:38.851765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:38.853360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e324dd-2737-4f2d-a717-7c0061803cde service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:38.863391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 708/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:39.673444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed878b3-2d39-49f4-a26a-dc45e06fd579 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:39.674336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed878b3-2d39-49f4-a26a-dc45e06fd579 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:39.674460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed878b3-2d39-49f4-a26a-dc45e06fd579 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:39.676734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:39.685025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f075363-06f2-4538-8091-e253b404b8fc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:39.685448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f075363-06f2-4538-8091-e253b404b8fc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:39.685632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f075363-06f2-4538-8091-e253b404b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:39.706235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f075363-06f2-4538-8091-e253b404b8fc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:39.707483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f075363-06f2-4538-8091-e253b404b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:39.716597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:39.867383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed878b3-2d39-49f4-a26a-dc45e06fd579 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:39.886977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed878b3-2d39-49f4-a26a-dc45e06fd579 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:39.894753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:39.895434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:39.895749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:39.896026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:39.904617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:39.906213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594b4db-254c-4df4-9734-5e71867b9508 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:39.916397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 728/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:40.731470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912645-aea3-47c4-a614-5175e7911794 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:40.732530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912645-aea3-47c4-a614-5175e7911794 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:40.732530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1912645-aea3-47c4-a614-5175e7911794 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:40.734619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:35:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:40.740092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb77c-6080-4ac0-9911-caa09258654e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:40.740561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb77c-6080-4ac0-9911-caa09258654e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:40.740664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb77c-6080-4ac0-9911-caa09258654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:40.760263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-31dbb77c-6080-4ac0-9911-caa09258654e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:40.761442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-31dbb77c-6080-4ac0-9911-caa09258654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:40.770848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:40.919450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912645-aea3-47c4-a614-5175e7911794 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:40.937021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1912645-aea3-47c4-a614-5175e7911794 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:40.943699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:40.944343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:40.944639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:40.944920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:40.954645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:40.956261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a1104-de1e-4bf3-9f72-c3b0727a9e45 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:40.965813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 711/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:42.887616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178018c-1851-469d-b68e-c70546070dda None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:42.888124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178018c-1851-469d-b68e-c70546070dda None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:42.888124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178018c-1851-469d-b68e-c70546070dda None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:42.889558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:35:42.896215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043acb-a33c-4434-b285-fa03965afe83 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:42.896678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043acb-a33c-4434-b285-fa03965afe83 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:42.896785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e043acb-a33c-4434-b285-fa03965afe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:42.924953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e043acb-a33c-4434-b285-fa03965afe83 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:42.926165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e043acb-a33c-4434-b285-fa03965afe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:42.935662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:42.977859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178018c-1851-469d-b68e-c70546070dda None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:42.980192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5178018c-1851-469d-b68e-c70546070dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:42.995603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178018c-1851-469d-b68e-c70546070dda None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:42.999747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:43.000383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:43.000654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:43.001060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:43.019021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:43.021417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af31eb-c265-4792-8fda-2795f771ef3c service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:43.036782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 731/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:45.249606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f03b-4297-485e-ab02-4aaa6c3bbf1f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:45.250227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f03b-4297-485e-ab02-4aaa6c3bbf1f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:45.250227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f03b-4297-485e-ab02-4aaa6c3bbf1f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:45.252119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:45.257504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dd64-9e4d-4697-8231-a4305e62d85f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:45.257953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dd64-9e4d-4697-8231-a4305e62d85f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:45.258062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75dd64-9e4d-4697-8231-a4305e62d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:45.277429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4a75dd64-9e4d-4697-8231-a4305e62d85f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:45.278603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4a75dd64-9e4d-4697-8231-a4305e62d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:45.287482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:45.326417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f03b-4297-485e-ab02-4aaa6c3bbf1f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:45.339816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f03b-4297-485e-ab02-4aaa6c3bbf1f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:45.346280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:45.347249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:45.347608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:45.347949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:45.364215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:45.366266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14728aa-0433-499b-98f4-a1034943d54e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:45.379688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 714/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:47.746503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756f1bd-c9c4-4e90-af12-ab6e4d1813b7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:47.746949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756f1bd-c9c4-4e90-af12-ab6e4d1813b7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:47.747060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756f1bd-c9c4-4e90-af12-ab6e4d1813b7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:47.748575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:47.754221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b74cd-f77a-4032-a9ee-2f3055ec1ae3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:47.754642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b74cd-f77a-4032-a9ee-2f3055ec1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:47.754766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b74cd-f77a-4032-a9ee-2f3055ec1ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:47.772789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-618b74cd-f77a-4032-a9ee-2f3055ec1ae3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:47.773978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-618b74cd-f77a-4032-a9ee-2f3055ec1ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:47.782033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:35:47.828656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756f1bd-c9c4-4e90-af12-ab6e4d1813b7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:47.841439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756f1bd-c9c4-4e90-af12-ab6e4d1813b7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:47.846413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:47.847065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:47.847374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:47.847644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:47.858142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:47.859636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7008b4-b5d1-4144-b192-f128e9dc0427 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:47.869279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 734/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:50.118625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ac9ad-48b9-4de3-ba4d-c5b9f98bacf8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:50.119406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ac9ad-48b9-4de3-ba4d-c5b9f98bacf8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:50.119526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ac9ad-48b9-4de3-ba4d-c5b9f98bacf8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:50.121956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:50.129768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1889a-d6d6-4fe2-999d-ebe073f1cf9e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:50.130227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1889a-d6d6-4fe2-999d-ebe073f1cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:50.130509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1889a-d6d6-4fe2-999d-ebe073f1cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:50.148994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-23b1889a-d6d6-4fe2-999d-ebe073f1cf9e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:50.150182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-23b1889a-d6d6-4fe2-999d-ebe073f1cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:50.159115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:50.201056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ac9ad-48b9-4de3-ba4d-c5b9f98bacf8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:50.221046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ac9ad-48b9-4de3-ba4d-c5b9f98bacf8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:50.225936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:50.226590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:50.226874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:50.227191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:50.238126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:50.239622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b725df-419a-4f2e-b015-dc3605ba9c57 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:50.249947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 717/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:52.372299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436ee9b-344c-4faf-8485-4047e624c38c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:52.372886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436ee9b-344c-4faf-8485-4047e624c38c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:52.372886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436ee9b-344c-4faf-8485-4047e624c38c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:52.374257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:52.380029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0f85-9f87-4f48-a53d-08ea3b4e68db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:52.380514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0f85-9f87-4f48-a53d-08ea3b4e68db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:52.380606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0f85-9f87-4f48-a53d-08ea3b4e68db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:52.401244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d98b0f85-9f87-4f48-a53d-08ea3b4e68db None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:52.402447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d98b0f85-9f87-4f48-a53d-08ea3b4e68db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:52.412150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:52.450998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436ee9b-344c-4faf-8485-4047e624c38c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:52.468960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436ee9b-344c-4faf-8485-4047e624c38c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:52.473684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:52.474282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:52.474571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:52.474839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:52.485194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:52.486796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb56171-8e93-4408-91b1-5d24cd5deb03 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:52.498334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 737/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:54.614191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ae599-1d3e-44e1-897a-30723172f1fa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:54.614813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ae599-1d3e-44e1-897a-30723172f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:54.614813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ae599-1d3e-44e1-897a-30723172f1fa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:54.616286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:54.622550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f023049-2ae2-47e4-bdbc-cb0996a955ab None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:54.623335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f023049-2ae2-47e4-bdbc-cb0996a955ab None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:54.623489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f023049-2ae2-47e4-bdbc-cb0996a955ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:54.644596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2f023049-2ae2-47e4-bdbc-cb0996a955ab None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:54.645804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f023049-2ae2-47e4-bdbc-cb0996a955ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:54.656123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:54.696744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ae599-1d3e-44e1-897a-30723172f1fa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:54.715192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ae599-1d3e-44e1-897a-30723172f1fa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:54.721808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:54.722457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:54.722773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:54.723040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:54.733025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:54.734465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96ae6c-f691-4669-af1b-c529f8f9183b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:54.744770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 720/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:56.829451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233e60-aeca-453d-b3cf-0ea463c88c24 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:56.830424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233e60-aeca-453d-b3cf-0ea463c88c24 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:56.830424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70233e60-aeca-453d-b3cf-0ea463c88c24 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:56.832821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:56.839139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2188-c410-43cb-93ee-62b890ed66be None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:56.839591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2188-c410-43cb-93ee-62b890ed66be None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:56.839705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2188-c410-43cb-93ee-62b890ed66be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:56.860715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-31cd2188-c410-43cb-93ee-62b890ed66be None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:56.862012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31cd2188-c410-43cb-93ee-62b890ed66be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:56.870886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:56.909145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70233e60-aeca-453d-b3cf-0ea463c88c24 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:56.922002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70233e60-aeca-453d-b3cf-0ea463c88c24 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:56.926284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:56.926937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:56.927277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:56.927560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:56.937274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:56.938782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dc9a-520f-4bcc-ae88-7218e8443b38 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:56.948558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 740/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:35:59.089686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3c8f-34e5-4a54-bb2f-8ee29ef47eac None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:59.090225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3c8f-34e5-4a54-bb2f-8ee29ef47eac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:59.090225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb3c8f-34e5-4a54-bb2f-8ee29ef47eac None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:59.091818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:35:59.097218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5508-22af-413f-8b22-6854b58951c7 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:59.097702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5508-22af-413f-8b22-6854b58951c7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:59.097810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5508-22af-413f-8b22-6854b58951c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:59.105026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=79 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:59.119937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-77bd5508-22af-413f-8b22-6854b58951c7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:35:59.121179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-77bd5508-22af-413f-8b22-6854b58951c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:35:59.131042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:35:59.172166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb3c8f-34e5-4a54-bb2f-8ee29ef47eac None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:35:59.181286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=73 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:35:59.190794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb3c8f-34e5-4a54-bb2f-8ee29ef47eac None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:35:59.194892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:35:59.195561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:35:59.195882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:35:59.196117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:35:59.205869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:35:59.207335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f858451-57bc-4a64-9e46-135f0d4fcfba service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:35:59.216834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 723/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:01.301869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d47093-397a-443f-bf16-0d965fb52e83 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:01.302474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d47093-397a-443f-bf16-0d965fb52e83 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:01.302474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d47093-397a-443f-bf16-0d965fb52e83 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:01.303857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:01.309292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d31-b98f-4bab-8512-7f2fd71f3993 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:01.309721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d31-b98f-4bab-8512-7f2fd71f3993 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:01.309833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f8d31-b98f-4bab-8512-7f2fd71f3993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:01.329374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-614f8d31-b98f-4bab-8512-7f2fd71f3993 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:01.330757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-614f8d31-b98f-4bab-8512-7f2fd71f3993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:01.339341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:01.375739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d47093-397a-443f-bf16-0d965fb52e83 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:01.388154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d47093-397a-443f-bf16-0d965fb52e83 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:01.392355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:01.393143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:01.393426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:01.393700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:01.403292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:01.404767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af296b20-2240-45c9-93c7-cb71d9c494f2 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:01.414565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 743/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:03.491418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d351c7-df11-4063-98d6-020eaaf4aafa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:03.491950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d351c7-df11-4063-98d6-020eaaf4aafa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:03.492036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d351c7-df11-4063-98d6-020eaaf4aafa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:03.494089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:36:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:03.501692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ccfee-a9bd-4a9b-925a-755e0e5f32d2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:03.502317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ccfee-a9bd-4a9b-925a-755e0e5f32d2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:03.502631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ccfee-a9bd-4a9b-925a-755e0e5f32d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:03.520595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e99ccfee-a9bd-4a9b-925a-755e0e5f32d2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:03.521801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e99ccfee-a9bd-4a9b-925a-755e0e5f32d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:03.529865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:03.566868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d351c7-df11-4063-98d6-020eaaf4aafa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:03.584399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d351c7-df11-4063-98d6-020eaaf4aafa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:03.590591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:03.591633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:03.592012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:03.592352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:03.604522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:03.606348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2afbb-e415-47f5-8282-d327eebfe18d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:03.616508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 726/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:13.515479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=11 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:13.611883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=14 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:23.829904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab657e-6cf4-4540-be84-ec607d8903e3 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.830814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab657e-6cf4-4540-be84-ec607d8903e3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.830814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab657e-6cf4-4540-be84-ec607d8903e3 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:23.833838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 745/1471] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:23.842203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596d120-19c4-46c2-92e3-73fc0a7ef1f1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.842631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596d120-19c4-46c2-92e3-73fc0a7ef1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.842771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596d120-19c4-46c2-92e3-73fc0a7ef1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:23.861341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8596d120-19c4-46c2-92e3-73fc0a7ef1f1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:23.862540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8596d120-19c4-46c2-92e3-73fc0a7ef1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:23.871668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 727/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:23.926501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba94-6ab3-4a9b-b4eb-94b4cc627d25 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.926816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba94-6ab3-4a9b-b4eb-94b4cc627d25 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.927133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba94-6ab3-4a9b-b4eb-94b4cc627d25 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:23.928556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 746/1473] 10.222.0.65 () {54 vars in 962 bytes} [Fri Jun 24 09:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:23.933718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573bbbd-b91c-4635-afff-10dd1d366cc7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.934006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573bbbd-b91c-4635-afff-10dd1d366cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.934281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573bbbd-b91c-4635-afff-10dd1d366cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:23.954534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a573bbbd-b91c-4635-afff-10dd1d366cc7 None None] MFA Rules not processed for user `01aa5644b3834b0d887bbfc854ebc8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:23.955756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a573bbbd-b91c-4635-afff-10dd1d366cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:23.964052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 728/1474] 10.222.0.65 () {58 vars in 1059 bytes} [Fri Jun 24 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:23.975021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddba94-6ab3-4a9b-b4eb-94b4cc627d25 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:23.988006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddba94-6ab3-4a9b-b4eb-94b4cc627d25 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:23.992061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:23.993172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:23.993533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:23.993886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:24.008466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:24.010536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f8c3c-30e9-49b9-a63e-ab2256a6ab23 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:24.025299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 747/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:26.184235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6555df-bd0a-4cb0-9c13-37f00b7f919c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:26.184810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6555df-bd0a-4cb0-9c13-37f00b7f919c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:26.184810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6555df-bd0a-4cb0-9c13-37f00b7f919c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:26.186215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 729/1476] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:26.191605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189360-bd17-496f-b403-c582acf810e1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:26.192120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189360-bd17-496f-b403-c582acf810e1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:26.192493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41189360-bd17-496f-b403-c582acf810e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:26.211404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-41189360-bd17-496f-b403-c582acf810e1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:26.212594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-41189360-bd17-496f-b403-c582acf810e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:26.221271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 748/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:26.246455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6555df-bd0a-4cb0-9c13-37f00b7f919c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:26.260223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6555df-bd0a-4cb0-9c13-37f00b7f919c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:26.265315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:26.265977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:26.266257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:26.266529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:26.267297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:26.269121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:36:26.269121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:36:26.269121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:36:26.270427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf712f-f62c-4fc6-91fd-960303602ef1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:26.290862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 730/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Fri Jun 24 09:36:26 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:36:28.360135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fdadb-1d19-44e0-96dc-bf288c381fbd None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:28.360744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fdadb-1d19-44e0-96dc-bf288c381fbd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:28.360744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fdadb-1d19-44e0-96dc-bf288c381fbd None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:28.362066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 749/1479] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:28.367735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef028b-9490-4055-9b79-868805e60d60 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:28.368192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef028b-9490-4055-9b79-868805e60d60 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:28.368296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef028b-9490-4055-9b79-868805e60d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:28.387438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-06ef028b-9490-4055-9b79-868805e60d60 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:28.388666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-06ef028b-9490-4055-9b79-868805e60d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:28.398647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 731/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:28.441787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fdadb-1d19-44e0-96dc-bf288c381fbd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:28.457530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fdadb-1d19-44e0-96dc-bf288c381fbd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:28.464038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:28.464690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:28.464970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:28.465249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:28.473950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:28.475381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d8249-53fb-45ef-a10a-84846b10fd77 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:28.485322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 750/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:30.707913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62720-0b3b-4209-85f9-ac3f65bab806 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:30.708573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62720-0b3b-4209-85f9-ac3f65bab806 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:30.708573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62720-0b3b-4209-85f9-ac3f65bab806 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:30.709920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 732/1482] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:30.715536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebb6b-691f-4fc6-91df-560200890a21 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:30.716020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebb6b-691f-4fc6-91df-560200890a21 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:30.716121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aebb6b-691f-4fc6-91df-560200890a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:30.735310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-59aebb6b-691f-4fc6-91df-560200890a21 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:30.736511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-59aebb6b-691f-4fc6-91df-560200890a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:30.745623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 751/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:30.781879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c62720-0b3b-4209-85f9-ac3f65bab806 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:30.793424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c62720-0b3b-4209-85f9-ac3f65bab806 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:30.797417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:30.798066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:30.798346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:30.798619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:30.807882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:30.809332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f8e14-503e-4701-a254-752327565894 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:30.819102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 733/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:32.880659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44653034-a028-4c83-ba00-3ceae1334a68 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:32.881256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44653034-a028-4c83-ba00-3ceae1334a68 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:32.881256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44653034-a028-4c83-ba00-3ceae1334a68 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:32.882925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 752/1485] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:32.888611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abf934-7625-45a2-b3f0-8daf070d57c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:32.889069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abf934-7625-45a2-b3f0-8daf070d57c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:32.889164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abf934-7625-45a2-b3f0-8daf070d57c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:32.907408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-58abf934-7625-45a2-b3f0-8daf070d57c3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:32.908599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58abf934-7625-45a2-b3f0-8daf070d57c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:32.916978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 734/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:36:32.956077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44653034-a028-4c83-ba00-3ceae1334a68 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:32.968982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44653034-a028-4c83-ba00-3ceae1334a68 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:32.973410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:32.974114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:32.974396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:32.974669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:32.984717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:32.986178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a1c7a-67ef-4562-a817-9f63a23eb591 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:32.996196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 753/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:35.214691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f79d-79f9-40d2-9601-0e28f5c82428 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:35.215291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f79d-79f9-40d2-9601-0e28f5c82428 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:35.215291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f79d-79f9-40d2-9601-0e28f5c82428 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:35.216664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 735/1488] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:35.222087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1af85b-3c71-4760-9a25-8c5c1f9835d4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:35.222604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1af85b-3c71-4760-9a25-8c5c1f9835d4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:35.222604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1af85b-3c71-4760-9a25-8c5c1f9835d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:35.241583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5a1af85b-3c71-4760-9a25-8c5c1f9835d4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:35.242797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5a1af85b-3c71-4760-9a25-8c5c1f9835d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:35.252136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 754/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:35.293093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82f79d-79f9-40d2-9601-0e28f5c82428 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:35.304813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82f79d-79f9-40d2-9601-0e28f5c82428 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:35.308748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:35.309397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:35.309685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:35.309963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:35.320676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:35.322139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd096736-1b86-44f4-a347-c16c5c462424 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:35.331864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 736/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:37.647169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03e2ef-1f52-4728-9a53-094242cb1ac5 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:37.647800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03e2ef-1f52-4728-9a53-094242cb1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:37.647800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03e2ef-1f52-4728-9a53-094242cb1ac5 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:37.649167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 755/1491] 10.222.0.65 () {54 vars in 926 bytes} [Fri Jun 24 09:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:36:37.655103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ccee-cc58-4215-a6fe-4b7924303d05 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:37.655688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ccee-cc58-4215-a6fe-4b7924303d05 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:37.655688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ccee-cc58-4215-a6fe-4b7924303d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:37.675188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e127ccee-cc58-4215-a6fe-4b7924303d05 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:36:37.676404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e127ccee-cc58-4215-a6fe-4b7924303d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:36:37.686057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 737/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Fri Jun 24 09:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:36:37.725468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03e2ef-1f52-4728-9a53-094242cb1ac5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:36:37.736596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03e2ef-1f52-4728-9a53-094242cb1ac5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:36:37.740258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:36:37.740932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:36:37.741219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:36:37.741503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:36:37.751569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:36:37.753039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca2d17-3c13-4dd4-bc4a-89aa6cb055f4 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:36:37.763445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 756/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:36:47.669658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:36:47.758729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=33 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:38:04.003228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.003819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.003819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.028234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:38:04.029453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:04.039670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 738/1494] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:38:04.049411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.049994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.049994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.070652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:38:04.071871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:04.080900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 757/1495] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:38:04.085872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.096769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fba91-268c-408a-a2e9-1648125a8fc7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.104152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.104809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.105090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.105357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.106088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:38:04.106399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.107469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.107469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:38:04.107469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:38:04.108372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.121656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 739/1496] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:38:04.129824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.139694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee92b48-adb4-4dd6-94ff-12263ee61124 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.144576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.145237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.145511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.145786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.146464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.148281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.148281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:38:04.148281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:38:04.150271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.192259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 758/1497] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:38:04.198479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.209277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8753d3-3456-4807-90dd-383558523631 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.277987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.278665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.278976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.279285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.279993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.281012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.281012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:38:04.281012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:38:04.282240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.304041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 740/1498] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:38:04.308926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.322757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e102187-db1b-4c07-a4a9-32057bef0d18 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.329996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.330676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.331357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.331357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.331954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.332984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.332984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:38:04.332984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:38:04.333873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.347920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 759/1499] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:38:04.352274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.364363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529960da-6831-4f17-a841-7ce719a8b9ce admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.370366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.371579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.371579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.371812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.372444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.373844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.373844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:38:04.373844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:38:04.374969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c7365-de7a-437d-ad6e-1392a08542c2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.389890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 741/1500] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:38:04.394451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.405374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975539-66af-4ecf-bc72-53335fabd8e9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.410702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.411673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.411972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.412237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] PATH_INFO: `/v3/projects/ae49c7ca0ce14e9d8a1ed26e4f93e9e1/users/be2b3096667c4827bf264c10f53465da/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.437414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae49c7ca0ce14e9d8a1ed26e4f93e9e1, user_id=be2b3096667c4827bf264c10f53465da, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.438488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.438488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:38:04.438488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:38:04.440480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cb30c-7f3d-44af-9e1d-87084e63639f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.456500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 760/1501] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:38:04 2022] PUT /identity/v3/projects/ae49c7ca0ce14e9d8a1ed26e4f93e9e1/users/be2b3096667c4827bf264c10f53465da/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:38:04.469470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a32e-56cc-4a07-bd63-bb1503d9fe73 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.469755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a32e-56cc-4a07-bd63-bb1503d9fe73 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.470038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5a32e-56cc-4a07-bd63-bb1503d9fe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.499519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e2f5a32e-56cc-4a07-bd63-bb1503d9fe73 None None] MFA Rules not processed for user `be2b3096667c4827bf264c10f53465da`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:38:04.500740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e2f5a32e-56cc-4a07-bd63-bb1503d9fe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:04.573401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 742/1502] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:38:04.587304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e346b8-1ab0-4c28-9c89-0d2b269e8927 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.587600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e346b8-1ab0-4c28-9c89-0d2b269e8927 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.587883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e346b8-1ab0-4c28-9c89-0d2b269e8927 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.589288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 761/1503] 10.222.0.65 () {54 vars in 963 bytes} [Fri Jun 24 09:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 09:38:04.594607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602b37-0acd-4c7a-ac44-39bb331d4083 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.594900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602b37-0acd-4c7a-ac44-39bb331d4083 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.595269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c602b37-0acd-4c7a-ac44-39bb331d4083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.614677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4c602b37-0acd-4c7a-ac44-39bb331d4083 None None] MFA Rules not processed for user `fd8aa4f4a5cb441cae26c8dafa27548c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:38:04.615921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c602b37-0acd-4c7a-ac44-39bb331d4083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:04.664115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:38:04.677789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 743/1504] 10.222.0.65 () {58 vars in 1060 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:38:04.687491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e346b8-1ab0-4c28-9c89-0d2b269e8927 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.699813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e346b8-1ab0-4c28-9c89-0d2b269e8927 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.703562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.704220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.704506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.704786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.713743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.715249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.724901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 762/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:38:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:38:04.769933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555f718-c4d8-4b9c-94cc-9d0379ccd0da None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.770223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555f718-c4d8-4b9c-94cc-9d0379ccd0da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.770507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555f718-c4d8-4b9c-94cc-9d0379ccd0da None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.771833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 744/1506] 10.222.0.65 () {52 vars in 829 bytes} [Fri Jun 24 09:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:38:04.776594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:04.840396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b8826-3d9d-43e3-869d-ea01526ad02c service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:04.845225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:04.846113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.846395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.846674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] PATH_INFO: `/v3/users/be2b3096667c4827bf264c10f53465da` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:04.847919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be2b3096667c4827bf264c10f53465da)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:04.848961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:04.848961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:38:04.848961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:38:04.850165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:04.922406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be2b3096667c4827bf264c10f53465da'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:04.942718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be2b3096667c4827bf264c10f53465da'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:04.943658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be2b3096667c4827bf264c10f53465da'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:04.954978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be2b3096667c4827bf264c10f53465da'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:04.966837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:38:04.971687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be2b3096667c4827bf264c10f53465da'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:04.981325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 763/1507] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:38:04 2022] DELETE /identity/v3/users/be2b3096667c4827bf264c10f53465da => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:38:04.986315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:04.986600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:04.986879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:05.005378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:38:05.006579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:05.068480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 745/1508] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:38:05.080301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:05.081850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:05.147106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f0c-f432-4cb0-9d29-2c3dcce9acd4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:05.152073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:05.152750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:05.153034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:05.153309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:05.163542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:05.165595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11d619-7061-4c23-8a7c-040b15d37daa service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:05.175609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 764/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:38:05.539943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:38:05.541430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:38:05.553502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19894d60-574e-47f8-8513-30f6448b6246 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:38:05.559275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:38:05.560296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:38:05.560582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:38:05.560845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] PATH_INFO: `/v3/projects/ae49c7ca0ce14e9d8a1ed26e4f93e9e1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:38:05.562132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae49c7ca0ce14e9d8a1ed26e4f93e9e1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:38:05.563227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:38:05.563227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:38:05.563227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:38:05.564446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:38:05.630646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae49c7ca0ce14e9d8a1ed26e4f93e9e1', 'request_id': 'req-68d4bde1-e091-438a-8d47-28833e5cbcac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:05.640937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae49c7ca0ce14e9d8a1ed26e4f93e9e1', 'request_id': 'req-68d4bde1-e091-438a-8d47-28833e5cbcac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:05.651149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae49c7ca0ce14e9d8a1ed26e4f93e9e1', 'request_id': 'req-68d4bde1-e091-438a-8d47-28833e5cbcac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:05.670921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] The token cache is being invalidate because project ae49c7ca0ce14e9d8a1ed26e4f93e9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:38:05.671981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:05.671981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-68d4bde1-e091-438a-8d47-28833e5cbcac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:38:05.673559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 746/1510] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:38:05 2022] DELETE /identity/v3/projects/ae49c7ca0ce14e9d8a1ed26e4f93e9e1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:38:15.171183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=29 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:38:15.667854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:39:45.868356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:45.869328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:45.869328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:45.871560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:39:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:39:45.877565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e0243-fa01-4dff-9577-db67fbf2c1b9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:45.878353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e0243-fa01-4dff-9577-db67fbf2c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:45.878518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e0243-fa01-4dff-9577-db67fbf2c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:45.898214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c5e0243-fa01-4dff-9577-db67fbf2c1b9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:45.899464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c5e0243-fa01-4dff-9577-db67fbf2c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:45.968591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:39:46.133058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:46.135332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:46.215817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a6a3b-996c-43f5-ad6f-6936b1c2c0bf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:46.221754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:46.222420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:46.222702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:46.223005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:46.231257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:46.232883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1d48a-ef88-4065-8f66-e4289eb7efff service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:46.242668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 766/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:39:47.092532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a755f7-cebf-466e-9ebe-14c7424134fa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:47.093121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a755f7-cebf-466e-9ebe-14c7424134fa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:47.093121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a755f7-cebf-466e-9ebe-14c7424134fa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:47.094471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:39:47.099762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf32d1-976a-41da-933d-2d38757c1ba1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:47.100230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf32d1-976a-41da-933d-2d38757c1ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:47.100334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf32d1-976a-41da-933d-2d38757c1ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:47.125496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-32cf32d1-976a-41da-933d-2d38757c1ba1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:47.126770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-32cf32d1-976a-41da-933d-2d38757c1ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:47.136573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:39:47.288260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a755f7-cebf-466e-9ebe-14c7424134fa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:47.313747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a755f7-cebf-466e-9ebe-14c7424134fa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:47.321275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:47.322007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:47.322269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:47.322548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:47.331124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:47.332805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae15e-be67-4514-b02d-5aa0d923199d service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:47.342573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 749/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:39:47.487532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea944b-ab51-4a47-b11f-db4b1eee6692 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:47.487795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea944b-ab51-4a47-b11f-db4b1eee6692 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:47.488078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea944b-ab51-4a47-b11f-db4b1eee6692 None None] PATH_INFO: `/v3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:47.490055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 768/1517] 10.222.0.47 () {54 vars in 925 bytes} [Fri Jun 24 09:39:47 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:39:47.496927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a02f-1476-4fad-a137-dfe55839c5e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:47.497220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a02f-1476-4fad-a137-dfe55839c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:47.497492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a02f-1476-4fad-a137-dfe55839c5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:47.639942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a704a02f-1476-4fad-a137-dfe55839c5e1 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:47.641093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a704a02f-1476-4fad-a137-dfe55839c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:47.715991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 750/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Fri Jun 24 09:39:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:39:51.503854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b058-3373-4b06-9d20-f9450df78391 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:51.504750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b058-3373-4b06-9d20-f9450df78391 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:51.504750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b058-3373-4b06-9d20-f9450df78391 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:51.507815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 769/1519] 10.222.0.65 () {54 vars in 960 bytes} [Fri Jun 24 09:39:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:39:51.512935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95874f-1018-42d1-b32c-13f27ccc14cd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:51.513448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95874f-1018-42d1-b32c-13f27ccc14cd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:51.513571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95874f-1018-42d1-b32c-13f27ccc14cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:51.532905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5c95874f-1018-42d1-b32c-13f27ccc14cd None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:51.534124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5c95874f-1018-42d1-b32c-13f27ccc14cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:51.545451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 751/1520] 10.222.0.65 () {58 vars in 1057 bytes} [Fri Jun 24 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:39:51.553895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b058-3373-4b06-9d20-f9450df78391 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:51.571219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b058-3373-4b06-9d20-f9450df78391 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:51.578602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:51.579327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:51.579614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:51.579935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:51.589755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:51.591957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2f64b-2334-438b-bafc-01b0b40307a9 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:51.604929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 770/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:39:58.603130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912c66-1bba-4ee0-8066-20eba8a30a22 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:58.603743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912c66-1bba-4ee0-8066-20eba8a30a22 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:58.603743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a912c66-1bba-4ee0-8066-20eba8a30a22 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:58.605084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:39:58.611528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88e3b5-93be-4604-9b70-2a4473676f0d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:58.612250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88e3b5-93be-4604-9b70-2a4473676f0d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:58.612587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88e3b5-93be-4604-9b70-2a4473676f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:58.643657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1a88e3b5-93be-4604-9b70-2a4473676f0d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:58.644892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1a88e3b5-93be-4604-9b70-2a4473676f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:58.653764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:39:58.812660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a912c66-1bba-4ee0-8066-20eba8a30a22 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:58.833179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a912c66-1bba-4ee0-8066-20eba8a30a22 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:58.840005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:58.840730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:58.841027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:58.841304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:58.849510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:58.851107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4b5b4-d18e-4580-af4f-f4fe3fd774bf service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:58.861698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 753/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:39:59.682988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec767ab-3f85-4c51-928b-42bf633e49fb None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:59.683647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec767ab-3f85-4c51-928b-42bf633e49fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:59.683647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec767ab-3f85-4c51-928b-42bf633e49fb None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:59.685547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 09:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:39:59.691311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047dc5-8bb9-4392-a3fe-87e49fd7466f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:59.691913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047dc5-8bb9-4392-a3fe-87e49fd7466f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:59.691913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047dc5-8bb9-4392-a3fe-87e49fd7466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:59.714877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c047dc5-8bb9-4392-a3fe-87e49fd7466f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:39:59.716777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c047dc5-8bb9-4392-a3fe-87e49fd7466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:39:59.726912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 09:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:39:59.884835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec767ab-3f85-4c51-928b-42bf633e49fb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:39:59.907491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec767ab-3f85-4c51-928b-42bf633e49fb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:39:59.914077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:39:59.914723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:39:59.915004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:39:59.915307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:39:59.923730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:39:59.925316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea011bf-9a2e-41bf-8ef3-054d43f4a486 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:39:59.935009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 773/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:03.828873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:03.829623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:03.829623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:03.831572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:40:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:40:03.837466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13fbe3-ce49-468b-8ce8-6919bd67e486 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:03.837913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13fbe3-ce49-468b-8ce8-6919bd67e486 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:03.838019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13fbe3-ce49-468b-8ce8-6919bd67e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:03.856733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bc13fbe3-ce49-468b-8ce8-6919bd67e486 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:40:03.857949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bc13fbe3-ce49-468b-8ce8-6919bd67e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:03.866723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:40:03.903152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:03.905472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:03.967225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e045c96-e990-4150-a1ba-ddd6f18bc435 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:03.971566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:03.972216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:03.972496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:03.972781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:03.982706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:03.984188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfec5c-2000-4b86-9c2c-d2d59df1ebff service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:03.989625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:40:03.996950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 756/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:06.156297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d279e-2e7c-4fb6-a812-fe6127f08153 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:06.156854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d279e-2e7c-4fb6-a812-fe6127f08153 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:06.156854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d279e-2e7c-4fb6-a812-fe6127f08153 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:06.158231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:40:06.164432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f0f2-7ca4-45b2-9b69-bab41d05f8be None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:06.165249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f0f2-7ca4-45b2-9b69-bab41d05f8be None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:06.165419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f0f2-7ca4-45b2-9b69-bab41d05f8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:06.191635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1896f0f2-7ca4-45b2-9b69-bab41d05f8be None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:40:06.192863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1896f0f2-7ca4-45b2-9b69-bab41d05f8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:06.201437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:40:06.248783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d279e-2e7c-4fb6-a812-fe6127f08153 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:06.261182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d279e-2e7c-4fb6-a812-fe6127f08153 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:06.265356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:06.266003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:06.266295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:06.266571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:06.276265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:06.278155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced5d14-d458-40bc-a927-13e1d57497ba service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:06.287902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 776/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:10.008127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1349-685e-4692-9549-4b616708afad None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:10.008690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1349-685e-4692-9549-4b616708afad None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:10.008690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1349-685e-4692-9549-4b616708afad None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:10.009974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:40:10.016346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d362d-e4de-4b71-aaea-60f2c870648f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:10.016808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d362d-e4de-4b71-aaea-60f2c870648f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:10.016905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d362d-e4de-4b71-aaea-60f2c870648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:10.041401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7f2d362d-e4de-4b71-aaea-60f2c870648f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:40:10.042643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7f2d362d-e4de-4b71-aaea-60f2c870648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:10.054251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:40:10.360428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1349-685e-4692-9549-4b616708afad None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:10.362415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a33d1349-685e-4692-9549-4b616708afad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:10.424594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1349-685e-4692-9549-4b616708afad None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:10.428920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:10.429561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:10.429845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:10.430127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:10.440492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:10.441916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdff02d-d6af-4caa-857f-97469dc6f989 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:10.453876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 759/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:12.502561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:12.503183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:12.503183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:12.504537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 09:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:40:12.510899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab00ea-eca3-425c-81d8-acbd8add42d7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:12.511656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab00ea-eca3-425c-81d8-acbd8add42d7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:12.511830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab00ea-eca3-425c-81d8-acbd8add42d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:12.539368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-99ab00ea-eca3-425c-81d8-acbd8add42d7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:40:12.540617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99ab00ea-eca3-425c-81d8-acbd8add42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:12.551547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 09:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:40:12.869895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:40:12.873032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:40:12.887441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e11358-50ee-4ebd-8092-f4d9165acd05 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:40:12.891273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:40:12.891944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:40:12.892255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:40:12.892534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:40:12.901567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:40:12.903015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce17e06-7740-477e-9b80-5ffe04ac7552 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:40:12.912738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 779/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:40:22.532541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=32 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:40:22.908749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=70 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:41.003827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.005001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.005557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.029530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.031268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.043541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 761/1540] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.053448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.054058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.054058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.090715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.092542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.107755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 780/1541] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.113184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.129556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4633c-c209-4552-a469-74379cc52c7b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.134280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.135255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.135679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.135953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.136690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:41.137003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.138633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.152544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 762/1542] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:41.161285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.176219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3824-9b86-4d49-ae08-013cd368ca5f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.185606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42545c01-d400-4368-a417-a33586690015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.186295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545c01-d400-4368-a417-a33586690015 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.186577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545c01-d400-4368-a417-a33586690015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.186848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42545c01-d400-4368-a417-a33586690015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.187579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42545c01-d400-4368-a417-a33586690015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.189505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42545c01-d400-4368-a417-a33586690015 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.236412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 781/1543] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:41.240873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.257696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69205d-35b8-4321-a80b-244e2baf9983 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.313497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.314195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.314697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.335868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.336583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.336871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.337147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.338362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.340187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa822050-355a-41f8-96ce-851a7b3ab02f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.362218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.367227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.385555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 782/1544] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.394590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 763/1545] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:41.398866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.403304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0544b9-5e93-4155-8191-6fd9f792c4ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.403595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0544b9-5e93-4155-8191-6fd9f792c4ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.403874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0544b9-5e93-4155-8191-6fd9f792c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.426144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d496b-fc8c-45b3-93bf-496e34c482df None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.446480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.447297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.447297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.447926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.448631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.450306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.455967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4b0544b9-5e93-4155-8191-6fd9f792c4ea None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.457177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b0544b9-5e93-4155-8191-6fd9f792c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.475113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 764/1546] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:41:41.476428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.476718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 783/1547] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:41.476854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.477044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.489492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.513727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a94bd-2a5c-4515-a07a-70498220ff7c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.519939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.521545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.539791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.540620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.540831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 765/1548] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.541197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.541297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.541999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.542628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.545528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4e25c-87e1-40c3-8d1a-2505e67cdd42 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.565981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db8b38-94e0-4d5c-95da-ecd6ebe45408 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.567611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 784/1549] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:41.568912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.569202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.569494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.591024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.592984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.593483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.594056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.595014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:41.595546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.597411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9352d51-2e42-47a4-87d2-2552def84157 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.614088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.615335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.620942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 766/1550] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:41.622880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15e12-d3e7-4d42-b1c5-7345bee3c55f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.623411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15e12-d3e7-4d42-b1c5-7345bee3c55f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.623951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15e12-d3e7-4d42-b1c5-7345bee3c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.625344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 785/1551] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.629931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.640653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0811df9-6386-4a83-9e32-64ce1b250fd0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.646141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.647588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.648096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.648598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] PATH_INFO: `/v3/projects/33dfd41a1d2747d0b41cf80a107c9e35/users/cd66084e257b4f65b74407899b8c9614/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.660823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-82a15e12-d3e7-4d42-b1c5-7345bee3c55f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.662302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82a15e12-d3e7-4d42-b1c5-7345bee3c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.667227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33dfd41a1d2747d0b41cf80a107c9e35, user_id=cd66084e257b4f65b74407899b8c9614, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.669947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01766bed-e09f-472e-92b8-c2a8c0790b27 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.671653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 767/1552] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.673518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.674120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.674322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.687888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 786/1553] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:41 2022] PUT /identity/v3/projects/33dfd41a1d2747d0b41cf80a107c9e35/users/cd66084e257b4f65b74407899b8c9614/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:41.693519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.694710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.697563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0927a-986c-4e8d-aa0e-bc1d684428fb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.698128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0927a-986c-4e8d-aa0e-bc1d684428fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.698502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0927a-986c-4e8d-aa0e-bc1d684428fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.717995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-22f0927a-986c-4e8d-aa0e-bc1d684428fb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.719255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-22f0927a-986c-4e8d-aa0e-bc1d684428fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.759334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 768/1554] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.760298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.773553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9de082-aafa-4f67-a0a6-41562ff0e9e4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.778111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.778848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.779060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.779407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.780049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.781075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:41.781075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:41:41.781075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:41:41.782260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.785467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 787/1555] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.786824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51020d08-9670-4f44-ade9-2ef0d0f434f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.787145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51020d08-9670-4f44-ade9-2ef0d0f434f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.787424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51020d08-9670-4f44-ade9-2ef0d0f434f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.805333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-51020d08-9670-4f44-ade9-2ef0d0f434f6 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.806616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-51020d08-9670-4f44-ade9-2ef0d0f434f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.814877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 788/1556] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.816175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330ba-f3aa-4cac-87d2-255f0972a4ec None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.816466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330ba-f3aa-4cac-87d2-255f0972a4ec None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.816775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330ba-f3aa-4cac-87d2-255f0972a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.821784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 769/1557] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:41.821784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:41.836536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a4c330ba-f3aa-4cac-87d2-255f0972a4ec None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.838004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a4c330ba-f3aa-4cac-87d2-255f0972a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.838199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:41.883714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927428bd-738d-4e64-8f30-ff3e62b0fed8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:41.888568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:41.889245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.889531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.889812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.890543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:41.890851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:41.892512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422c708-16ec-465a-a1b1-a994eeaa48f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:41.897845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 789/1558] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.899755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.899755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.899755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.911767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 770/1559] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:41.911767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ef21-e8f6-424f-80dc-fdf831dab17c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.911767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ef21-e8f6-424f-80dc-fdf831dab17c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.911767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ef21-e8f6-424f-80dc-fdf831dab17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.914069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:41.925268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7cd2ef21-e8f6-424f-80dc-fdf831dab17c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.926476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7cd2ef21-e8f6-424f-80dc-fdf831dab17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.929647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] MFA Rules not processed for user `cd66084e257b4f65b74407899b8c9614`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.930852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.935446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 771/1560] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.936684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65347-8712-49b2-983d-fe5f78920818 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.937052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65347-8712-49b2-983d-fe5f78920818 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.937299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd65347-8712-49b2-983d-fe5f78920818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.955749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0cd65347-8712-49b2-983d-fe5f78920818 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.956956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0cd65347-8712-49b2-983d-fe5f78920818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.965322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 772/1561] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.966527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:41.966812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:41.967098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:41.985196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:41.986419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:41.994635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 773/1562] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:41.995519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.002044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 790/1563] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:42.002910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.005172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292f4bf-618b-4563-9214-37a8be813e46 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.009977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.010665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.010952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.011277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.012008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:42.012314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.013918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55618dd3-6414-437a-8644-47c553f1dfe8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.014075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2af1-7c1b-437d-8172-c711698497f2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.020801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.022118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.022118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.022335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.023069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.024508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:42.024508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:41:42.024508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:41:42.026343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.026712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 774/1564] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:42.027973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608933bb-ed87-437e-a57d-639047b508f3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.028257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608933bb-ed87-437e-a57d-639047b508f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.028515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608933bb-ed87-437e-a57d-639047b508f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.048921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-608933bb-ed87-437e-a57d-639047b508f3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:42.050100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-608933bb-ed87-437e-a57d-639047b508f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:42.050521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 791/1565] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.054640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.061410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 775/1566] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:42.062651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.062937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.063214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.068166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9bf25-8cd0-4e00-9eb0-72186a988070 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.073421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.074096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.074386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.074670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.075400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.077350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.081248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:42.082425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:42.091107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 776/1567] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:42.091900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.105283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d9409-13c7-4a11-aaee-992380372b20 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.107724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 792/1568] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.108999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.113474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.114137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.114416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.114693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.115638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:42.115947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.118073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.131145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 777/1569] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:42.132274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.142582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e973d-f8a2-466b-9484-e3407afc9cd4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.174448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e091-6ab1-4ce6-8e01-51f1d61c84f2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.180338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.182049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.182333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.182647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.183417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:42.183724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.184764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:42.184764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:41:42.184764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:41:42.185658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.197374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 793/1570] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:42.198163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.201782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.202845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.203498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.203780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.204481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.207143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.208667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39086-2418-4028-805a-8cb305aef0e6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.213823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.214551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.214848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.215162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.215853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.217783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.235138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 778/1571] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.235990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.250682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 794/1572] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.251421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.276347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:42.283555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:42.303173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9d067-a684-41bf-8777-f8520709eb0c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.315645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a2042-5cfc-4bff-9b74-8e349cd4ac96 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.320494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.321168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.321448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.321783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.322504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:42.322809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.324491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.337412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 795/1573] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:42.344002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.354602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796b40-a9ad-43b3-a631-a16d58913c3a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.360153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.360828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.361094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.361368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.362116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:42.362420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.364070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.370204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.370887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.371199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.371476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.372214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.373814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.383340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 796/1574] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:42.383340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.383857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 779/1575] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.384655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.394330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ff13d-702a-4326-b572-f5d4e2037e6d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.394536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477508b5-510b-4063-b2ac-2ffc82e26db9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.399040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.399204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.399764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.400031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.400294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.400464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.400553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.400643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.400991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.401167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.402884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.403197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.428423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 797/1576] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.428423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.439465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 780/1577] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.439720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560221-8496-4b9f-acbe-9e49d2c3398a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.439880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.494082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.494525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.494805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.495096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.495930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.497863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.498279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84daec-4c94-46c9-b46d-c2bb33ffe2ed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.503119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.503958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.504236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.504508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.505183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.507113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.527746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 781/1578] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.528057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 798/1579] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.528450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.529241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.538810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d18c06-c300-437c-9207-ed8d9a442832 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.585089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:42.599799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d980754-7258-4ff8-9645-73f244d2043a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.600252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.600947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.601226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.601490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.602189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.604198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.605376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.606056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.606340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.606619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.607351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.609642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.630653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 782/1580] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.630653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.637753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e275a8d-f0e0-4b7a-93e6-fcf0a6832f25 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.644715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 799/1581] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.644715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.645655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.646327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.646615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.646923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.647643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.649330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:42.649600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.679566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 783/1582] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.680282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.708081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c32a4-ed61-4e64-a3d0-7b71e72b5094 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.745420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63310abf-c1c2-477b-ab19-7424cbcdb11b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.750545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.751422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.751736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.752020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.752758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.754415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.765490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 784/1583] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.765688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.766151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.766270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.766554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.766834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.767818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.769643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.777024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff847-b0a3-4ab5-8a4b-b08e45265164 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.781387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 800/1584] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.782358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.784366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.785100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.785425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.785699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.786386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.788409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.792695 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8dd34-4971-46b5-97bf-a7a2907b0b83 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.797264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.798104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.798390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.798670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.799762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.801118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.812494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 801/1585] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.816922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 785/1586] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.817778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.817911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.826712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5bad-5eb5-4205-af53-78f883f52d04 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.827383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f829f-c0ae-4208-af44-8b269481f403 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.831930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.832605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.832719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.832873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.833148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.833392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.833703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.833842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.834006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.834760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.835849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.836456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.847912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 786/1587] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.848715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.858927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc98c-d2fe-4f71-a84c-18f623370b64 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.860405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 802/1588] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.861364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.863960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.864642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.864920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.865193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.865909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.867868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.872241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c54cdf-8617-4f55-94e5-75b7456bac07 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.877810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.879358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.879358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.879358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.879946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.881991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.888583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 787/1589] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.889767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.902166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e80b-6993-49b9-90ef-89a871edad86 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.907441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 803/1590] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:42 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:42.907441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.907680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.907680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.907883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.908099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.908893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.910572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.915294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb013b1d-4722-44b2-86d1-22431074fcf7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.920222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.921150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.921460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.921734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] PATH_INFO: `/v3/projects/4290bd187dc948bc9f1dc5a8cc272f0b/users/f1722779f6b74af59fa05a2b63890b2d/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.923258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 788/1591] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:42.924512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.935994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344457ca-aede-46be-b0b9-b50fe7c7ac61 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:42.938347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4290bd187dc948bc9f1dc5a8cc272f0b, user_id=f1722779f6b74af59fa05a2b63890b2d, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.940713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:42.941053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.941601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:42.941883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:42.942286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] PATH_INFO: `/v3/projects/f4275e6efe4949bca6073f308ad4c061/users/74a0aa6368874de0a8790001fbe8dd1c/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:42.955448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 804/1592] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:42 2022] PUT /identity/v3/projects/4290bd187dc948bc9f1dc5a8cc272f0b/users/f1722779f6b74af59fa05a2b63890b2d/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:42.956374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.957944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4275e6efe4949bca6073f308ad4c061, user_id=74a0aa6368874de0a8790001fbe8dd1c, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:42.958992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:42.958992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:41:42.958992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:41:42.960975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:42.976507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 789/1593] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:42 2022] PUT /identity/v3/projects/f4275e6efe4949bca6073f308ad4c061/users/74a0aa6368874de0a8790001fbe8dd1c/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:42.977402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:42.996457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.009533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.021406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef7341-bc57-473a-b00e-09535a446b3c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.043055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07fd40-5c01-4050-bd7c-22396e88c746 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.047659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.048335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.048620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.048896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.049623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.051292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.061286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 790/1594] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.061919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.071391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091f82-5d84-4636-847b-df333104f181 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.076202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.076892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.077174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.077446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.078167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.079588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.080052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.080293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.080581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.080861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.081603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.083238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.090337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 791/1595] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.091201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.093943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 805/1596] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.097431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.100595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fbd20-725f-4a5a-9e22-b7239402c071 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.105419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.106069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.106351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.106640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.106964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d118-c82c-408d-9be6-c8852fa34769 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.107373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.109330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.112098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.112748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.113038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.113313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.114041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.115653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec2055-aa7d-410e-88c4-0356d50a29f9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.119794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 792/1597] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.123512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.125989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 806/1598] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.130084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.130380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.130661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.133210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e729a-e7f1-4240-b257-cc7a5d3fbe78 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.137879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.138753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.139050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.139358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] PATH_INFO: `/v3/projects/eaa9106852b94278b74b5deb38bde1e4/users/59c16a4cf1a847728944b5a062b457bd/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.148329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:43.149606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:43.155103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa9106852b94278b74b5deb38bde1e4, user_id=59c16a4cf1a847728944b5a062b457bd, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.157777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.158162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 807/1599] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:43.158915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.170267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcd4d-be82-4e06-8bb9-d235dd4ec00a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.173685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 793/1600] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/eaa9106852b94278b74b5deb38bde1e4/users/59c16a4cf1a847728944b5a062b457bd/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.174519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.226597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.227211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.227523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.227952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.228713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.230378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.240806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 808/1601] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.241864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26364efd-d5f9-4f70-ab07-3c0593b9c4f5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.244228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.246950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.247887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.248169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.248441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] PATH_INFO: `/v3/projects/9f07f0fbe3b449faaf6f6c26604aebba/users/7447a1eeb2304c80993aa63409e7820b/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.254525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec436b1-50f3-49da-a548-cafbe80288c9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.259859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.260601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.260956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.261305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.262075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.263364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f07f0fbe3b449faaf6f6c26604aebba, user_id=7447a1eeb2304c80993aa63409e7820b, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.263718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.266114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.274313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 809/1602] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.275358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.282173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 794/1603] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/9f07f0fbe3b449faaf6f6c26604aebba/users/7447a1eeb2304c80993aa63409e7820b/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.283287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.295010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4baca5-58d8-4b5d-9e64-2531b76bdfd6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.339171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.346849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167932e-5a49-4af5-917d-dc81e467b7f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.351761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.352117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.352467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.352749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.352906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.353028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.353118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.353402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.353990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.354245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.355683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.355991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.357759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.366770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 795/1604] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.367864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.368686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 810/1605] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.369668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.382447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc8d50-2e28-4c39-b1ee-01cbc2510c8f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.387351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.388035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.388322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.388602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.389340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.390972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388264bb-24f5-47d4-b629-bd04167ebf74 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.401900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 796/1606] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:43.403253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321f86-f9b0-4c9f-b1c4-46a0b9abf412 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.403541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321f86-f9b0-4c9f-b1c4-46a0b9abf412 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.403826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321f86-f9b0-4c9f-b1c4-46a0b9abf412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.429025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427fa73-e52c-478a-bcf1-13ba07f9193d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.430244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-70321f86-f9b0-4c9f-b1c4-46a0b9abf412 None None] MFA Rules not processed for user `59c16a4cf1a847728944b5a062b457bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:43.431914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-70321f86-f9b0-4c9f-b1c4-46a0b9abf412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:43.432908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.433542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.433828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.434110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.443730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.445184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.455541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 811/1607] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:43.456422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.467674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f663b-a890-475e-8f2b-6a9123d4a9d7 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.473647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.474565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.474843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.475143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] PATH_INFO: `/v3/projects/f4275e6efe4949bca6073f308ad4c061/users/74a0aa6368874de0a8790001fbe8dd1c/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.477444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4275e6efe4949bca6073f308ad4c061, user_id=74a0aa6368874de0a8790001fbe8dd1c, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.480338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.492385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 812/1608] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/f4275e6efe4949bca6073f308ad4c061/users/74a0aa6368874de0a8790001fbe8dd1c/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.493450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.504400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 797/1609] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:43.505934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.506225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.506505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.546031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] MFA Rules not processed for user `7447a1eeb2304c80993aa63409e7820b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:43.550040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:43.585929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa815-ccf8-4311-95fb-47590626b5ee admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.592357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.593122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.593541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.593838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] PATH_INFO: `/v3/projects/30033ce42b5a43b78dde38163831cea2/users/fd136fcda9a04a16b258301a86bde8ef/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.615186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30033ce42b5a43b78dde38163831cea2, user_id=fd136fcda9a04a16b258301a86bde8ef, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.618814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.629742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 798/1610] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:43.630589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.641275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 813/1611] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/30033ce42b5a43b78dde38163831cea2/users/fd136fcda9a04a16b258301a86bde8ef/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.642067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.648808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f604f42-9770-4a17-9387-906d07df8d1c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.717925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.719003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.719455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.719953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] PATH_INFO: `/v3/projects/f64a8807499a44ababb8bb037fc3d195/users/544cf118cc9e4aed8e7ec7e9dfe023bc/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.720395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.727408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fb0db-f2e5-45e5-96eb-d48ce6f650d7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.736342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.737114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:43.737487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.737893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.738174 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] PATH_INFO: `/v3/projects/c7d337515eaf429e8ec6fb1e918dac56/users/f1c24c05d8034f34a501f7b9d72e7be7/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.749203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64a8807499a44ababb8bb037fc3d195, user_id=544cf118cc9e4aed8e7ec7e9dfe023bc, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.752764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.757016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d337515eaf429e8ec6fb1e918dac56, user_id=f1c24c05d8034f34a501f7b9d72e7be7, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.762498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fede25-66bb-460d-b70f-146a095d6704 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.772664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 799/1612] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/f64a8807499a44ababb8bb037fc3d195/users/544cf118cc9e4aed8e7ec7e9dfe023bc/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.774329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.781800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 814/1613] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/c7d337515eaf429e8ec6fb1e918dac56/users/f1c24c05d8034f34a501f7b9d72e7be7/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.783337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.783623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.783900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.824829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] MFA Rules not processed for user `74a0aa6368874de0a8790001fbe8dd1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:43.826059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:43.854292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab0d21-c335-402a-9c8d-04c3efee25ca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.921333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.922503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.922900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.923308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] PATH_INFO: `/v3/projects/42946f76423740028ae43dc6a49cbbda/users/d5e6d9fad06449c1b43254df442d224f/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.927939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 815/1614] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:43.929076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.945026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218ce8d-01a0-4946-ac8b-545950d30e14 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:43.945823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42946f76423740028ae43dc6a49cbbda, user_id=d5e6d9fad06449c1b43254df442d224f, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.948779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.951223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:43.952564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.953225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.953720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:43.955019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:43.957065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae40c057-230d-45b6-b86d-f29712b780aa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:43.969353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 800/1615] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:43 2022] PUT /identity/v3/projects/42946f76423740028ae43dc6a49cbbda/users/d5e6d9fad06449c1b43254df442d224f/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:43.969353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:43.989886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 816/1616] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:43.991254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62994-a817-4511-9995-b36aa710d96b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:43.991703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62994-a817-4511-9995-b36aa710d96b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:43.991993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd62994-a817-4511-9995-b36aa710d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.011425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bcd62994-a817-4511-9995-b36aa710d96b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:44.012816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bcd62994-a817-4511-9995-b36aa710d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.047210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90932149-03de-452d-b88f-3ba55574479e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.071480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:44.080014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 817/1617] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:44.081513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d78bfd-e56b-4eee-a087-dc01711f6b63 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.081795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d78bfd-e56b-4eee-a087-dc01711f6b63 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.082135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d78bfd-e56b-4eee-a087-dc01711f6b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.100556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-50d78bfd-e56b-4eee-a087-dc01711f6b63 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:44.101873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-50d78bfd-e56b-4eee-a087-dc01711f6b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.108898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.109657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.110010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.110682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.111165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.111657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 818/1618] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:44.112822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bac751-5ab6-46ef-bba8-960693b20505 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.113177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bac751-5ab6-46ef-bba8-960693b20505 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.113521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bac751-5ab6-46ef-bba8-960693b20505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.113622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.145111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-43bac751-5ab6-46ef-bba8-960693b20505 None None] MFA Rules not processed for user `f1c24c05d8034f34a501f7b9d72e7be7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:44.146462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43bac751-5ab6-46ef-bba8-960693b20505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.153630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 801/1619] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:43 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:44.154598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.177118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:44.219989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bbf1f-407a-40bb-8890-e6a96b258910 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.230151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.231461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.232320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.232604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.243156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 819/1620] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:44.245624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.245916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.246195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.272276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:44.273390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.345729 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.347771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.350742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 820/1621] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:44.351886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.364986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e624a0-0a73-40e8-b1ba-d750148d5b1c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.371042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 802/1622] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:44.372139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.381692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.382210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.382499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.382843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.385407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.387437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.425603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 821/1623] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/users => generated 309 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:44.426502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.498444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834b23-acd5-47d7-bd28-45f9ca56c2a7 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.506326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.507030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.508794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.509113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.518547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e62cf8-64e5-4cc5-a7aa-1e7e61409c97 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.522629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.523346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.523673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.524032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.530779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.533449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.537118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.539375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3d59-61db-40d0-a57f-fba32d916e07 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.547378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 803/1624] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:44.549062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.552260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 822/1625] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:44.553677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.553969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.554245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.555886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 823/1626] 10.222.0.65 () {54 vars in 961 bytes} [Fri Jun 24 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:44.557717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.560414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:44.561437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.568870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba98aa-9868-4bd8-be71-381211ec8499 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.577759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979464f-7eea-431a-9b37-1b782b0a56b1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.581280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.581788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.582504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.582793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.583064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.584828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.586500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.586878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.588161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.594627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.596184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.598069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.614352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 824/1627] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:44.616141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.631414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c5bff-638f-44ae-98cc-0a22ccba9f86 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.638658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 804/1628] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:44.639388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.640583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.641149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.641602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.642623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.643025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.644853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cfe9b-4cf7-4c88-ad5c-03d2d14594b5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.654600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fc61-ce98-4aad-b5cb-0dcb65cab56b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.659135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.659903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.660278 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.661984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.662357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 825/1629] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:44.664703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.665256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.666884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.693342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:44.695039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:44.726636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:44.743850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.746580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.761531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 805/1630] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:44.762718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.788546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d67ce-dfec-461f-a26e-7b8a1c976167 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.802638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.803624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.805888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.806391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.808313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.817137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.860323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 806/1631] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:44.862410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.880850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 826/1632] 10.222.0.65 () {58 vars in 1058 bytes} [Fri Jun 24 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:44.882775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.888747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788edaf-3974-4500-ac0b-999e53e365ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.900422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.901694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.904358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.904757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] PATH_INFO: `/v3/projects/be3b9ebe2e7a4708a2811f1c72620591/users/f3ab137eb33f4b6da6f2090621244d1a/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.907842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b8fde-8a58-458a-a87b-7027264c040a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:44.923108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:44.924171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:44.925868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:44.927579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] PATH_INFO: `/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f/users/2beca5686bd0482588f1cd20c3157211/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:44.946361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3b9ebe2e7a4708a2811f1c72620591, user_id=f3ab137eb33f4b6da6f2090621244d1a, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.951410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.960735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc5a2d28ff1482592bfb8a49ccc1f7f, user_id=2beca5686bd0482588f1cd20c3157211, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:44.964697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:44.981734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 807/1633] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:44 2022] PUT /identity/v3/projects/be3b9ebe2e7a4708a2811f1c72620591/users/f3ab137eb33f4b6da6f2090621244d1a/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:44.983536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:44.988678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 827/1634] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:44 2022] PUT /identity/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f/users/2beca5686bd0482588f1cd20c3157211/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:44.998542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.092332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff028199-bd55-456d-856a-37e050e89e03 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.099575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.100546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.100921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.101479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.102336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.104780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.121948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 828/1635] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:45.123139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.137929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb7df-f131-44db-9154-f0dab047235f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.144971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.146563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.146563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.146759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.147636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.150118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.165239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c0a18-3939-4af2-8ea2-9063f474a208 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.171548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 829/1636] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:45.172833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.185035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760942d8-cb73-411a-a360-0041c51394f9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.191252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.193259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.193687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.194112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] PATH_INFO: `/v3/projects/be3b9ebe2e7a4708a2811f1c72620591/users/f3ab137eb33f4b6da6f2090621244d1a/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.196372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3b9ebe2e7a4708a2811f1c72620591, user_id=f3ab137eb33f4b6da6f2090621244d1a, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.199965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.231885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 830/1637] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:45 2022] PUT /identity/v3/projects/be3b9ebe2e7a4708a2811f1c72620591/users/f3ab137eb33f4b6da6f2090621244d1a/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:45.231885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.273203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:45.320149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.322994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.322994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.322994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.394948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c174d3-a869-4727-97dd-581580c8e995 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.403243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.406888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.408462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.408978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] PATH_INFO: `/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f/users/2beca5686bd0482588f1cd20c3157211/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.412261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc5a2d28ff1482592bfb8a49ccc1f7f, user_id=2beca5686bd0482588f1cd20c3157211, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.423667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1aea6-98c8-4370-a6e8-975d75ec69b3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.435641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:45.445119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 831/1638] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:45 2022] PUT /identity/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f/users/2beca5686bd0482588f1cd20c3157211/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:45.461210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.463306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.463794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.468143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.469806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f900d3a-2c8d-47c3-9039-7092c22b8065 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.498410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 808/1639] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:45.501805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.502097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.502443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.508942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] MFA Rules not processed for user `f3ab137eb33f4b6da6f2090621244d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:45.510917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:45.580157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] MFA Rules not processed for user `2beca5686bd0482588f1cd20c3157211`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:45.582258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:45.640962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 832/1640] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:45.686713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.788385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 809/1641] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:45.817868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:45.894780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757a9dc-3622-497e-8b35-e7cdece78ee9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.903764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.906864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.907173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.907468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.927284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.929753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a22c4-74ec-480c-b8a0-387c0a7cd0d6 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.949044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305530-a140-4a38-97ae-0590385cfe22 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:45.956081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 833/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:45 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:45.957410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:45.958263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:45.958526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:45.958800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:45.971534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:45.977303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368970b-74a9-42cc-babb-27f0ad0a82ad service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:45.994917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 810/1643] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:46.051441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d794-0ad2-4f00-9287-15e377c13c72 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.051802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d794-0ad2-4f00-9287-15e377c13c72 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.052179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540d794-0ad2-4f00-9287-15e377c13c72 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:46.054306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 834/1644] 10.222.0.65 () {54 vars in 912 bytes} [Fri Jun 24 09:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:46.060040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.060333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.060611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:46.098507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:46.099177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:46.117250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:46.200279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d794-0ad2-4f00-9287-15e377c13c72 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:46.311925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 811/1645] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:46.313821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540d794-0ad2-4f00-9287-15e377c13c72 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:46.325953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:46.329250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:46.329511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.329714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.329891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] PATH_INFO: `/v3/users/59c16a4cf1a847728944b5a062b457bd` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:46.335367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c16a4cf1a847728944b5a062b457bd)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:46.338606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:46.353740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260d99f-7a54-401a-80f4-3d7448abb9d4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:46.363950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:46.365037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.365037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.365438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:46.394557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:46.411461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a589-0477-4bbc-acea-4395a7ee2bf5 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:46.434566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:46.439128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 812/1646] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:46.467974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c16a4cf1a847728944b5a062b457bd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:46.500569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c16a4cf1a847728944b5a062b457bd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:46.503296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c16a4cf1a847728944b5a062b457bd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:46.521973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c16a4cf1a847728944b5a062b457bd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:46.535396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c16a4cf1a847728944b5a062b457bd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:46.561431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 835/1647] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:46 2022] DELETE /identity/v3/users/59c16a4cf1a847728944b5a062b457bd => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:46.565996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.566382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.566776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:46.620783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:46.624214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:46.768218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 813/1648] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:46.800322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:46.802517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:46.979313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828842e-8c4f-4eb3-97cc-74d99a6ff252 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:46.993649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:46.994561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:46.994849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:46.995241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:47.022722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:47.025344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc287a5-4ace-4516-ae63-b196fe00e72c service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:47.048339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 836/1649] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:47.894280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:47.895899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:47.918210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dcfa2-c606-44e4-856c-9da48fe7effb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:47.932014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:47.932827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:47.933114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:47.933394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] PATH_INFO: `/v3/users/7447a1eeb2304c80993aa63409e7820b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:47.938571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7447a1eeb2304c80993aa63409e7820b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:47.939615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:47.939615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 24 09:41:47.939615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: Jun 24 09:41:47.940865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:48.144104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7447a1eeb2304c80993aa63409e7820b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:48.178124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7447a1eeb2304c80993aa63409e7820b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:48.199691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7447a1eeb2304c80993aa63409e7820b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:48.201524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7447a1eeb2304c80993aa63409e7820b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:48.204891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e3b4b2b-96f3-4970-a025-41f646b34f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7447a1eeb2304c80993aa63409e7820b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:48.235533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1d31e-4fbe-4891-aa42-681ac47e729e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.235533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1d31e-4fbe-4891-aa42-681ac47e729e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.235533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1d31e-4fbe-4891-aa42-681ac47e729e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.237997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 837/1650] 10.222.0.65 () {54 vars in 961 bytes} [Fri Jun 24 09:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:48.247670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6e48b-dcbc-4320-a14c-beb705a7497c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.247962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6e48b-dcbc-4320-a14c-beb705a7497c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.248246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6e48b-dcbc-4320-a14c-beb705a7497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.279149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 814/1651] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:47 2022] DELETE /identity/v3/users/7447a1eeb2304c80993aa63409e7820b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:48.285999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.286297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f3c6e48b-dcbc-4320-a14c-beb705a7497c None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:48.287649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f3c6e48b-dcbc-4320-a14c-beb705a7497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:48.287900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.288435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.317476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:48.397490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:48.406541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:48.520004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 838/1652] 10.222.0.65 () {58 vars in 1058 bytes} [Fri Jun 24 09:41:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:48.526326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47732934-2520-46ad-aadd-7abc7646b875 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.527127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47732934-2520-46ad-aadd-7abc7646b875 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.527127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47732934-2520-46ad-aadd-7abc7646b875 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.529397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 839/1653] 10.222.0.65 () {54 vars in 912 bytes} [Fri Jun 24 09:41:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:48.535027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.535373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.535652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.548486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 815/1654] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:48.584886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:48.588120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:48.589881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:48.591334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:48.624523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 840/1655] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:41:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:48.641334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:48.643402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:48.726538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:48.746581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e4e2-d82a-4f89-9bfd-5c8b811b863b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:48.761550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:48.762015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.762311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.762593 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.781915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:48.783554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:48.800873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e09d-6173-4f21-a91e-861ed05ca8fb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:48.806427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 816/1656] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:41:48.810724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:48.811632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:48.812838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:48.815200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:48.844678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:48.849602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:48.880875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 841/1657] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:49.017529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:49.021395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:49.095109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3a7e-b919-4a1a-9fc9-90fde1e09d62 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:49.115261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:49.125354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:49.125354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:49.125354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] PATH_INFO: `/v3/projects/eaa9106852b94278b74b5deb38bde1e4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:49.129909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa9106852b94278b74b5deb38bde1e4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:49.136025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:49.298793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa9106852b94278b74b5deb38bde1e4', 'request_id': 'req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.327649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa9106852b94278b74b5deb38bde1e4', 'request_id': 'req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.338536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa9106852b94278b74b5deb38bde1e4', 'request_id': 'req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.371872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] The token cache is being invalidate because project eaa9106852b94278b74b5deb38bde1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:41:49.373103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.375750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.402024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 817/1658] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:49 2022] DELETE /identity/v3/projects/eaa9106852b94278b74b5deb38bde1e4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:49.402395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:49.416097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1753e0-2d4c-42cf-b2a8-4e1263c462c6 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:49.575736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:49.577589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:49.578432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:49.578842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] PATH_INFO: `/v3/users/f3ab137eb33f4b6da6f2090621244d1a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:49.580682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ab137eb33f4b6da6f2090621244d1a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:49.583026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:49.718516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ab137eb33f4b6da6f2090621244d1a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.761401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ab137eb33f4b6da6f2090621244d1a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.767453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ab137eb33f4b6da6f2090621244d1a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.789372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ab137eb33f4b6da6f2090621244d1a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.799723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ab137eb33f4b6da6f2090621244d1a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:49.815054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 842/1659] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:49 2022] DELETE /identity/v3/users/f3ab137eb33f4b6da6f2090621244d1a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:41:50.482676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:50.484651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:50.511313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b33a4-5d7e-4478-a2dd-d82fc3c4c7f8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:50.580259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:50.671381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:50.672637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:50.673132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:50.673715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] PATH_INFO: `/v3/projects/9f07f0fbe3b449faaf6f6c26604aebba` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:50.676022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f07f0fbe3b449faaf6f6c26604aebba)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:50.678204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:50.679570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:50.680674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:50.701591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad229-732a-45e1-9ed9-4529c8d11235 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:50.716099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:50.717077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:50.717367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:50.717693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] PATH_INFO: `/v3/projects/be3b9ebe2e7a4708a2811f1c72620591` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:50.720437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3b9ebe2e7a4708a2811f1c72620591)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:50.721861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 09:41:50.721861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Jun 24 09:41:50.721861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: Jun 24 09:41:50.725005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:50.797722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f07f0fbe3b449faaf6f6c26604aebba', 'request_id': 'req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.813937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f07f0fbe3b449faaf6f6c26604aebba', 'request_id': 'req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.833670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f07f0fbe3b449faaf6f6c26604aebba', 'request_id': 'req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.866658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3b9ebe2e7a4708a2811f1c72620591', 'request_id': 'req-f3e36c0f-e14e-4492-9143-5976ef1f9f48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.877327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:50.893265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3b9ebe2e7a4708a2811f1c72620591', 'request_id': 'req-f3e36c0f-e14e-4492-9143-5976ef1f9f48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.894996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] The token cache is being invalidate because project 9f07f0fbe3b449faaf6f6c26604aebba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:41:50.900735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.906146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.907859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3b9ebe2e7a4708a2811f1c72620591', 'request_id': 'req-f3e36c0f-e14e-4492-9143-5976ef1f9f48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.914826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 818/1660] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:50 2022] DELETE /identity/v3/projects/9f07f0fbe3b449faaf6f6c26604aebba => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:50.918636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:50.925986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:50.928218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] The token cache is being invalidate because project be3b9ebe2e7a4708a2811f1c72620591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:41:50.928675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.930125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f3e36c0f-e14e-4492-9143-5976ef1f9f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:50.938144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 843/1661] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:50 2022] DELETE /identity/v3/projects/be3b9ebe2e7a4708a2811f1c72620591 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:50.939893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:50.940330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:50.940872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.004804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:51.006448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:51.165370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 844/1662] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:51.166476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:51.167879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad2e9a-f3d1-48d3-9a45-88ed01c0d15b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.172543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:51.182428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.184775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.185068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.185352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] PATH_INFO: `/v3/users/2beca5686bd0482588f1cd20c3157211` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.190052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2beca5686bd0482588f1cd20c3157211)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.195695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.208412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd4a8b-9e98-4995-8944-0e3412737f61 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.217885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.219178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.220590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.221035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] PATH_INFO: `/v3/users/f1c24c05d8034f34a501f7b9d72e7be7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.225151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c24c05d8034f34a501f7b9d72e7be7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.228016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.315194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2beca5686bd0482588f1cd20c3157211'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.334832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2beca5686bd0482588f1cd20c3157211'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.336136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c24c05d8034f34a501f7b9d72e7be7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.355507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2beca5686bd0482588f1cd20c3157211'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.356926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2beca5686bd0482588f1cd20c3157211'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.358358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-341deba4-0eb5-45d6-baf2-d3c79d3ddd58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2beca5686bd0482588f1cd20c3157211'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.359933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:51.372738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c24c05d8034f34a501f7b9d72e7be7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.374551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c24c05d8034f34a501f7b9d72e7be7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.396349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c24c05d8034f34a501f7b9d72e7be7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.401020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 819/1663] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:50 2022] DELETE /identity/v3/users/2beca5686bd0482588f1cd20c3157211 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:51.404559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-32d51230-04c5-47b0-96e9-97eba62f89e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c24c05d8034f34a501f7b9d72e7be7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:51.404821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.405121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.405408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.417867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 845/1664] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:51 2022] DELETE /identity/v3/users/f1c24c05d8034f34a501f7b9d72e7be7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:51.422356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.422640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.422924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.456198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:51.457418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:51.468031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:51.469308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:51.556400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:51.597871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 846/1665] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:51.635033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:51.635260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 820/1666] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:51.638354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:51.639420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:51.662932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9118e2-da78-479a-b399-d938f570ad75 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.671387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.673012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.673355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.673745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.675300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:51.676085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.678029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.716046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 821/1667] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:51.717162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:51.750287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4712-beeb-4057-8baf-c598b6082939 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.765744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.767771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.768272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86e5a6-e6fd-4348-bc87-cd63106412b0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.768747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.768747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.769919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.772126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.778584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.779500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.780219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.780219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.800288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.801792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.831696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 822/1668] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:51 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:51.835959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:51.841888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 847/1669] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:41:51.932835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf8649-d9c2-4cfa-bc24-fcbcdedbfea5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:51.938543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:51.939388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:51.940135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:51.940135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:51.941291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:51.948279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:51.987123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 823/1670] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:51 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:51.988280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:52.011320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118623ed-9adf-4e10-a404-f86fdb0891c5 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:52.024475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:52.026218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.026218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.026218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.029110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:52.035152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:52.057553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 848/1671] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:41:52.059012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:52.070807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17e0a-5d4b-4102-9396-145e34a9fd13 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:52.084354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f973c-8738-425c-824b-631367092583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:52.085084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f973c-8738-425c-824b-631367092583 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.085404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f973c-8738-425c-824b-631367092583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.085697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f973c-8738-425c-824b-631367092583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.086450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f973c-8738-425c-824b-631367092583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:52.088453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f973c-8738-425c-824b-631367092583 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:52.111943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 824/1672] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:52.116765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:52.138515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d81717-82ee-4b49-b7da-cd76d8868a63 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:52.167591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:52.169137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.169306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.169545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] PATH_INFO: `/v3/projects/844021669aec4b5e978867d600376349/users/9d5a6beda1a94737bdf44f58a506efe4/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.251029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] RBAC: Authorizing `identity:create_grant(project_id=844021669aec4b5e978867d600376349, user_id=9d5a6beda1a94737bdf44f58a506efe4, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:52.254570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:52.285237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 849/1673] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:52 2022] PUT /identity/v3/projects/844021669aec4b5e978867d600376349/users/9d5a6beda1a94737bdf44f58a506efe4/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:52.298533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692883-9ba0-4ac0-913a-2c4bc1c315e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.299694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692883-9ba0-4ac0-913a-2c4bc1c315e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.300716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692883-9ba0-4ac0-913a-2c4bc1c315e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.360337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c5692883-9ba0-4ac0-913a-2c4bc1c315e1 None None] MFA Rules not processed for user `9d5a6beda1a94737bdf44f58a506efe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:52.361764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5692883-9ba0-4ac0-913a-2c4bc1c315e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:52.408148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:52.548140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 825/1674] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:52.549422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:52.688345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1663-3be4-4ae2-8585-04f4007bb918 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.688865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1663-3be4-4ae2-8585-04f4007bb918 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.688865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1663-3be4-4ae2-8585-04f4007bb918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.785758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-87fb1663-3be4-4ae2-8585-04f4007bb918 None None] MFA Rules not processed for user `f1722779f6b74af59fa05a2b63890b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:52.787017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-87fb1663-3be4-4ae2-8585-04f4007bb918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:52.800164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f958-ae63-4b49-af08-1fc16877442f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:52.828348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:52.829239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:52.829800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:52.831043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:52.834888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:52.837836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:52.909467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 850/1675] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:52 2022] POST /identity/v3/projects => generated 363 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:52.916932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.003750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 826/1676] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:53.015687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.015977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.016411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.034899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:53.056646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] MFA Rules not processed for user `f1722779f6b74af59fa05a2b63890b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:53.058121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.060687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd0037-4bc1-4afb-b85f-8c55d2c4ce50 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.075952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.077417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.077819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.078362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.079213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.083412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.126617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 851/1677] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:52 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:53.128206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.149047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11075fed-87ba-4611-b3f9-70cdf6e05c4a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.161414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.163307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.164235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.165780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.167587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.178068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.197548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 827/1678] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:53.199631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.200758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 852/1679] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:41:53.201265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.205074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.231440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6cdfd-625c-42b1-bf82-ec5a6d69e1ac None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.240791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.242545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.242720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.243027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] PATH_INFO: `/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.247671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc5a2d28ff1482592bfb8a49ccc1f7f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.252454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296818c-8a38-4bd3-8d73-a3b10e7246f8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.252609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.272958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.274244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.275615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.275830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] PATH_INFO: `/v3/projects/079713664fd5473c8e27d2ce47a3e30c/users/7212feb08e544b0ebe37adebcca973b2/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.309221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079713664fd5473c8e27d2ce47a3e30c, user_id=7212feb08e544b0ebe37adebcca973b2, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.312130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.335491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 853/1680] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:53 2022] PUT /identity/v3/projects/079713664fd5473c8e27d2ce47a3e30c/users/7212feb08e544b0ebe37adebcca973b2/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:53.337367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.340434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.349263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:53.360071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc5a2d28ff1482592bfb8a49ccc1f7f', 'request_id': 'req-01d3eb98-fc6e-400e-a2bf-15f356ce69be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:53.376398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc5a2d28ff1482592bfb8a49ccc1f7f', 'request_id': 'req-01d3eb98-fc6e-400e-a2bf-15f356ce69be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:53.385886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc5a2d28ff1482592bfb8a49ccc1f7f', 'request_id': 'req-01d3eb98-fc6e-400e-a2bf-15f356ce69be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:53.423839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] The token cache is being invalidate because project 5fc5a2d28ff1482592bfb8a49ccc1f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:41:53.424728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:53.426248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:53.435738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 828/1681] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:53 2022] DELETE /identity/v3/projects/5fc5a2d28ff1482592bfb8a49ccc1f7f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:41:53.436879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.446180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.500188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f34878-39c0-4d88-94de-fedbf2569d68 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.560673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3eb98-fc6e-400e-a2bf-15f356ce69be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.567483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.568086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.568518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.569550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.570852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.573215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c7985-5387-41e0-aba9-f7a39ad811cd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.602243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 829/1682] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:53.605745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.606199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.606648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.646639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:53.648262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.668550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 830/1683] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:53.669551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.680903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.681880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e2e54-b560-4d47-8faf-731f7d010095 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:53.681987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.682235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.682623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.685206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.690047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:53.703341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.704095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.704512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] PATH_INFO: `/v3/projects/079713664fd5473c8e27d2ce47a3e30c/users/7212feb08e544b0ebe37adebcca973b2/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.707563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=079713664fd5473c8e27d2ce47a3e30c, user_id=7212feb08e544b0ebe37adebcca973b2, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.711240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f8dff-84e0-41e6-806a-9e4769083ccc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.736074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 831/1684] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:53 2022] PUT /identity/v3/projects/079713664fd5473c8e27d2ce47a3e30c/users/7212feb08e544b0ebe37adebcca973b2/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:53.739449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.739449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.739739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.799431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:53.803294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:53.805488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.846657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:53.848761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d76bb3-3793-4840-b6a1-c2c5565bd679 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:53.874820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 854/1685] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:53.902588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:53.902858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:53.904994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:53.975113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 832/1686] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:53.977941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:53.979667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] MFA Rules not processed for user `7212feb08e544b0ebe37adebcca973b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:53.983279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:53.985686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.007391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fac8f-bb9b-4bb7-816c-71f0ee9e0e42 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.015953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.017081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.017456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.017990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] PATH_INFO: `/v3/projects/c7d337515eaf429e8ec6fb1e918dac56` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.019864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d337515eaf429e8ec6fb1e918dac56)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.022214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.089764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:54.095933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d337515eaf429e8ec6fb1e918dac56', 'request_id': 'req-23c59a52-dd93-4536-a1f9-475448462389'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:54.115374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d337515eaf429e8ec6fb1e918dac56', 'request_id': 'req-23c59a52-dd93-4536-a1f9-475448462389'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:54.129677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d337515eaf429e8ec6fb1e918dac56', 'request_id': 'req-23c59a52-dd93-4536-a1f9-475448462389'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:54.162590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] The token cache is being invalidate because project c7d337515eaf429e8ec6fb1e918dac56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:41:54.164695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:54.166675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23c59a52-dd93-4536-a1f9-475448462389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:54.167283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 855/1687] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:54.168234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:54.172780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 833/1688] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:53 2022] DELETE /identity/v3/projects/c7d337515eaf429e8ec6fb1e918dac56 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:54.177520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.177807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.178090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.196818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02c07d-1df4-43a1-a0bc-cd5f31fc73da None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.222047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:54.223751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.355669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.358777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.359092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.359367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.361115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:54.361446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.369042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307487d-6668-453a-8085-f152fa539bfb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.395459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 834/1689] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:54.397774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:54.400842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 856/1690] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:54.402030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.407296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.407586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.408839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.458564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:54.462741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.492831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 857/1691] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:54.493647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:54.505564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.534858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072089f-e2b1-41c0-ae16-30abc7ee885a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.540677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16489-a1f6-4e9f-bf33-151c047b0b31 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.553325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.554213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.554641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.556089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.559249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:54.564414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.565364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.567041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.567604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.569905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.573259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.674090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 858/1692] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:54 2022] POST /identity/v3/projects => generated 393 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:54.675726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:54.778405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:54.788536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.790542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.834637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 835/1693] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:54.835640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:54.890810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25978a-05ce-4160-b137-882c1c6cd1f9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.899149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd75e-2f03-49b9-bf6c-d57453eeb6c6 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:54.903129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41127935-0708-4267-9677-154a183ff15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.904209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127935-0708-4267-9677-154a183ff15b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.904703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127935-0708-4267-9677-154a183ff15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.905164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127935-0708-4267-9677-154a183ff15b admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.907342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41127935-0708-4267-9677-154a183ff15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:41:54.907855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41127935-0708-4267-9677-154a183ff15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.909939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41127935-0708-4267-9677-154a183ff15b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.946490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:54.946490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.946490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.946490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.946999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:54.950404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a66ef-93d2-46d2-9039-5f6eba2a94dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:54.972794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 859/1694] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:41:54.976547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38365864-b14e-405f-a78a-5dbd289ed158 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:54.976845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38365864-b14e-405f-a78a-5dbd289ed158 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:54.977123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38365864-b14e-405f-a78a-5dbd289ed158 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:54.987048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 860/1695] 10.222.0.65 () {54 vars in 912 bytes} [Fri Jun 24 09:41:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:54.987568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38365864-b14e-405f-a78a-5dbd289ed158 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.018405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 836/1696] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:54 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:55.019948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.022106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38365864-b14e-405f-a78a-5dbd289ed158 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.022641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.022858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.040411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.041286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.041575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.041851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.050447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.050801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:55.053399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.125121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 861/1697] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:54 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:55.126408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.164316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:55.169995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:55.315062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a06ce-87aa-40c1-8609-169c46ca4854 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.335353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.336096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.336384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.336658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.343300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.344082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 837/1698] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:41:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:55.348147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.349204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.383031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b5da5-faac-4ea8-8b03-2493fc217dce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.386681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 862/1699] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:55.387461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.398394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.399102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.399398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.400148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.400871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.403226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.412486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d0869-2024-4403-941b-bbd3cd0b77e1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.426201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.426201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.426201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.426201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.426488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.426488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.444361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 838/1700] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:55 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:55.448493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.454317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 863/1701] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:55.458166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.469158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb370-9652-42ee-9c82-714948778d85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.475656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd18c-420f-4575-81fc-0b81ada5c2c0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.480173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.481672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.483429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.483429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.484793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.488544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.489270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.491261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.492132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.492442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] PATH_INFO: `/v3/projects/421819eead3a48ea8ee00f8d778cf602/users/229ef186ef0b437e8c9136563ecea01a/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.513494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 839/1702] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:55.516625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.528638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421819eead3a48ea8ee00f8d778cf602, user_id=229ef186ef0b437e8c9136563ecea01a, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.532143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.538557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d34d3-a763-46c7-b94c-29c4c4fcb91f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.548157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.549904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.550475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.551025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.558540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.566103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc31c7-0b5c-4009-9b93-6b0f5c57795b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.567423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 864/1703] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:55 2022] PUT /identity/v3/projects/421819eead3a48ea8ee00f8d778cf602/users/229ef186ef0b437e8c9136563ecea01a/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:55.568812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.575349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:55.598510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 840/1704] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:55.602727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.603291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.604797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.674845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] MFA Rules not processed for user `229ef186ef0b437e8c9136563ecea01a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:55.680360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:55.723614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec231b82-ea43-43fb-97e7-dcfaaf557553 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:55.738216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:55.739331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:55.739831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:55.740453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:55.758803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:55.762460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:55.788138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 865/1705] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:55.789533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.807838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:55.873898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 841/1706] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:55.873898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:55.879843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.007563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9e8e4-4421-4a5c-90da-40e845a671c3 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.026590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:56.030565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.033716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.037554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.037883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] PATH_INFO: `/v3/projects/a21dba8e7c77448e8c60b133231597c3/users/0f533825847c402899836f97e38c2740/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.081256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8db13-1890-4e74-9043-f7d3febb0b03 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.094352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21dba8e7c77448e8c60b133231597c3, user_id=0f533825847c402899836f97e38c2740, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.098622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.109586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.110403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.110650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.110932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] PATH_INFO: `/v3/users/9d5a6beda1a94737bdf44f58a506efe4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.121083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5a6beda1a94737bdf44f58a506efe4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.126117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 866/1707] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:55 2022] PUT /identity/v3/projects/a21dba8e7c77448e8c60b133231597c3/users/0f533825847c402899836f97e38c2740/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:56.127840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.130433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.249832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2269ff-79ef-4632-a5e1-94657ad0cd4c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.250064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5a6beda1a94737bdf44f58a506efe4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:56.270591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5a6beda1a94737bdf44f58a506efe4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:56.276424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.280223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.280611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.280904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.282241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.284875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad75af6-fd64-42f8-8bdf-136f5b686980 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.290802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5a6beda1a94737bdf44f58a506efe4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:56.291536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5a6beda1a94737bdf44f58a506efe4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:56.292171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-59020f1d-0f08-45d9-b8e1-7cb9eff82871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5a6beda1a94737bdf44f58a506efe4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:41:56.355121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 842/1708] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:41:55 2022] DELETE /identity/v3/users/9d5a6beda1a94737bdf44f58a506efe4 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:56.356607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.358881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.359216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.371825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 867/1709] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/projects => generated 393 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:56.373672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.374744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.377947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.421602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:56.422963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.454678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] MFA Rules not processed for user `0f533825847c402899836f97e38c2740`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:56.456013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.506487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 868/1710] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:56.507375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.508758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.524375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afb261-f9c1-410a-b525-cea8c171797a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.530458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.531117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.531412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.531691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.533473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.536636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.550427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:56.576102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 869/1711] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/users => generated 303 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:56.580254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.596765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57063bdc-9f31-4d62-8454-989ffa2e9962 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.619130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 843/1712] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:56.620077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.620804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.621090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.621367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.622112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.624562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.624692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.626840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.639258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 870/1713] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:56.643378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.659444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d95314-c218-41c5-9c4b-c61787aed51f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.663903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978146a-6802-4242-b1d8-c2383e10c47c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.668451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.669333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.670018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.670305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.670584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.671239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.671339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.671529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.671812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] PATH_INFO: `/v3/projects/98fcd55e45544fb2851eff516f27b25d/users/2a450a3ee8c040358143867fcef44588/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.673334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.708210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fcd55e45544fb2851eff516f27b25d, user_id=2a450a3ee8c040358143867fcef44588, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.717674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.727392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 844/1714] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:56.731619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.770295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 871/1715] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:56 2022] PUT /identity/v3/projects/98fcd55e45544fb2851eff516f27b25d/users/2a450a3ee8c040358143867fcef44588/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:56.771199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.791849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:56.800633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:56.831946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31eee7-7b24-4414-9464-be1ca476b2de admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.922698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:56.924456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:56.925136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:56.925777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:56.926869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:56.930032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:56.970180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 845/1716] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:41:56 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:41:56.971722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:56.991172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa2cd5-47f5-4fae-b565-2bced36c2414 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:56.999048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.000009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.000714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.001368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.002329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.005736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890addd-a419-44e5-86d2-40a212bc5fe3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.014407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.015345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.015453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.015760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.017445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.057265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.058579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.058817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 846/1717] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:57.062517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.095838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e911bbe-38c1-4608-8500-e8bb84241b02 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.098605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 872/1718] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:57.099712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.112216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.114574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.115698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.117269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.118948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.128875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.129655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069359b-85de-4e12-a2b3-3ac2a4a1bcc6 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.143928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.145021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.145470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.147182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] PATH_INFO: `/v3/projects/98fcd55e45544fb2851eff516f27b25d/users/2a450a3ee8c040358143867fcef44588/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.148650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 847/1719] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:57.149968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.161271 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc258e-a8d3-4821-85c0-b220bdb1708a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.161735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fcd55e45544fb2851eff516f27b25d, user_id=2a450a3ee8c040358143867fcef44588, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.164769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15fa28-96ca-4249-b290-c5eeb3965415 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.167023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.168088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.168437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.168785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] PATH_INFO: `/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9/users/77b0215d958844b3b241482de0d1bf63/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.184578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 873/1720] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:57 2022] PUT /identity/v3/projects/98fcd55e45544fb2851eff516f27b25d/users/2a450a3ee8c040358143867fcef44588/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:57.198809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.199392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.199823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.199918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f94830bf8934c1da29a16d2a7ad21b9, user_id=77b0215d958844b3b241482de0d1bf63, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.203375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.232933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 848/1721] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:57 2022] PUT /identity/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9/users/77b0215d958844b3b241482de0d1bf63/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:57.234777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.271867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] MFA Rules not processed for user `2a450a3ee8c040358143867fcef44588`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:57.273423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:57.423905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 874/1722] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:57.438638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.446438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:57.490656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feee949-827b-49d6-8c42-91d688f8e2de admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.503960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.505584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.505972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.507009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.509492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.512395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fb9f5-18c3-4c59-87ff-96db6cfb2d4d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.549006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 849/1723] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:41:57.550539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a30a-9515-4a03-8ee9-2cad665af7d5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.553366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a30a-9515-4a03-8ee9-2cad665af7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.553725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a30a-9515-4a03-8ee9-2cad665af7d5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.558468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 850/1724] 10.222.0.65 () {54 vars in 960 bytes} [Fri Jun 24 09:41:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:41:57.559574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a30a-9515-4a03-8ee9-2cad665af7d5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:57.582468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:57.597830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a30a-9515-4a03-8ee9-2cad665af7d5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.616849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.619679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.622187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.622374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] PATH_INFO: `/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9/users/77b0215d958844b3b241482de0d1bf63/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.630206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f94830bf8934c1da29a16d2a7ad21b9, user_id=77b0215d958844b3b241482de0d1bf63, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.638111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9017c-74bc-4c0a-8dbe-b27c5e5338a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:57.649268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26f29-5fab-405f-aa7f-c86306a90bda None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:57.667903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 851/1725] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:41:57 2022] PUT /identity/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9/users/77b0215d958844b3b241482de0d1bf63/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:41:57.672562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f846ea-da33-4cd7-9a5a-942290295ed2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.673254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f846ea-da33-4cd7-9a5a-942290295ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.673547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f846ea-da33-4cd7-9a5a-942290295ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.722602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-24f846ea-da33-4cd7-9a5a-942290295ed2 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:57.726118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-24f846ea-da33-4cd7-9a5a-942290295ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:57.768214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:57.807928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:57.808727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.809034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.809321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.879127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 852/1726] 10.222.0.65 () {58 vars in 1057 bytes} [Fri Jun 24 09:41:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:57.881143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:57.881445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:57.881728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:57.936187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] MFA Rules not processed for user `77b0215d958844b3b241482de0d1bf63`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:41:57.937434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:57.980186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:57.981695 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:58.016133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 875/1727] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:58.026821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:58.077545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6c06-a849-4f12-8309-c572ea8d21cd service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:58.100007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:58.100924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:58.101351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:58.101768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:58.110385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:58.127148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 853/1728] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:41:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:41:58.142678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:58.144814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:58.153811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:58.172719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 876/1729] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:58.183420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae19d2-d0f2-40b4-8c35-831e5eb4907b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:58.202963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:58.203677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:58.203966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:58.208723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:58.209295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:58.211711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:58.229796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ce56-7c8a-45d3-a717-3843b83e4766 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:58.237861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:58.239926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:58.240315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:58.240736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:58.241171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:58.243710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:58.249891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:58.280283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 854/1730] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:58.439931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:58.442942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:58.469979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 877/1731] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:41:58.709637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:58.727283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64d358-b46b-4fa0-8390-eca51ea63456 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:58.731840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:58.732514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:58.733159 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:58.733626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:58.952240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:58.953822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:41:58.991475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 855/1732] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:41:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:41:59.665862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:41:59.668070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:41:59.751631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:41:59.848984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16f78c-5895-4403-9a08-79f8caa41566 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:41:59.863931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:41:59.865411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:41:59.865921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:41:59.866423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] PATH_INFO: `/v3/projects/844021669aec4b5e978867d600376349` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:41:59.869946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=844021669aec4b5e978867d600376349)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:41:59.885791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:00.038913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844021669aec4b5e978867d600376349', 'request_id': 'req-4ed60d75-0447-4af6-9dc2-5931f1d68166'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.049361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844021669aec4b5e978867d600376349', 'request_id': 'req-4ed60d75-0447-4af6-9dc2-5931f1d68166'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.063434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844021669aec4b5e978867d600376349', 'request_id': 'req-4ed60d75-0447-4af6-9dc2-5931f1d68166'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.093624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] The token cache is being invalidate because project 844021669aec4b5e978867d600376349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:00.094322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.095521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.102385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 878/1733] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:41:59 2022] DELETE /identity/v3/projects/844021669aec4b5e978867d600376349 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:00.102711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:00.124382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d872-d423-472c-81cb-74f0c1dc2673 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:00.189319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:00.317880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:00.319018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:00.319339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:00.319619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] PATH_INFO: `/v3/users/7212feb08e544b0ebe37adebcca973b2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:00.323109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7212feb08e544b0ebe37adebcca973b2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:00.327930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:00.483651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7212feb08e544b0ebe37adebcca973b2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.516984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7212feb08e544b0ebe37adebcca973b2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.548697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7212feb08e544b0ebe37adebcca973b2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.550589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7212feb08e544b0ebe37adebcca973b2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.552370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96de309c-aaa9-4064-be08-5736ebc8d695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7212feb08e544b0ebe37adebcca973b2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:00.608777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 856/1734] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:00 2022] DELETE /identity/v3/users/7212feb08e544b0ebe37adebcca973b2 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:02.096860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:02.100126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:02.142245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed60d75-0447-4af6-9dc2-5931f1d68166 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:02.189103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:02.189814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:02.189814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:02.249757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:02.250996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:02.381411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 857/1735] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:02.408743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:02.409976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:02.410434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:02.410710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] PATH_INFO: `/v3/projects/079713664fd5473c8e27d2ce47a3e30c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:02.413095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079713664fd5473c8e27d2ce47a3e30c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:02.417787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:02.515870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:02.517746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:02.546557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8ce6-53b1-42a9-8fe9-653d4246c7ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:02.567225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:02.568223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:02.571311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:02.572467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] PATH_INFO: `/v3/users/0f533825847c402899836f97e38c2740` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:02.574075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f533825847c402899836f97e38c2740)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:02.576409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:02.589959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079713664fd5473c8e27d2ce47a3e30c', 'request_id': 'req-3c1514a6-d9c1-433e-8245-383d3f0b8425'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.623953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079713664fd5473c8e27d2ce47a3e30c', 'request_id': 'req-3c1514a6-d9c1-433e-8245-383d3f0b8425'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.631403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:02.645639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079713664fd5473c8e27d2ce47a3e30c', 'request_id': 'req-3c1514a6-d9c1-433e-8245-383d3f0b8425'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.652761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:02.670082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] The token cache is being invalidate because project 079713664fd5473c8e27d2ce47a3e30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:02.670550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.672704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.679295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 879/1736] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:02 2022] DELETE /identity/v3/projects/079713664fd5473c8e27d2ce47a3e30c => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:02.679295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:02.679646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f533825847c402899836f97e38c2740'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.680550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:02.698228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f533825847c402899836f97e38c2740'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.724818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f533825847c402899836f97e38c2740'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.726353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f533825847c402899836f97e38c2740'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.727819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6b527896-e77e-4a8d-8910-d48c89e3b71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f533825847c402899836f97e38c2740'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:02.780266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 858/1737] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:02 2022] DELETE /identity/v3/users/0f533825847c402899836f97e38c2740 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:02.782428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ce5f-169c-4c14-a684-a1f40f2b3652 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:02.783902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ce5f-169c-4c14-a684-a1f40f2b3652 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:02.792655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ce5f-169c-4c14-a684-a1f40f2b3652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:02.833775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8017ce5f-169c-4c14-a684-a1f40f2b3652 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:02.835420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8017ce5f-169c-4c14-a684-a1f40f2b3652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:02.867226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1514a6-d9c1-433e-8245-383d3f0b8425 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:02.987933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 859/1738] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:02.989362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e270f-6ff6-4279-b1b7-740157c04ab2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:02.990355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e270f-6ff6-4279-b1b7-740157c04ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:02.990355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e270f-6ff6-4279-b1b7-740157c04ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.033153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9f5e270f-6ff6-4279-b1b7-740157c04ab2 None None] MFA Rules not processed for user `d5e6d9fad06449c1b43254df442d224f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:03.034793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9f5e270f-6ff6-4279-b1b7-740157c04ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.060357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.061416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.061871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.065540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.073627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.151915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 860/1739] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:03.153475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b131bb-f4ad-4f67-b3b3-ef6a38199af0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.153926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b131bb-f4ad-4f67-b3b3-ef6a38199af0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.154320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b131bb-f4ad-4f67-b3b3-ef6a38199af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.212796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-60b131bb-f4ad-4f67-b3b3-ef6a38199af0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:03.214297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60b131bb-f4ad-4f67-b3b3-ef6a38199af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.224789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 861/1740] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:03.228783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0466bd-f68d-4efe-86ae-2b91dac3436c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.233237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0466bd-f68d-4efe-86ae-2b91dac3436c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.233873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0466bd-f68d-4efe-86ae-2b91dac3436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.259354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:03.261125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:03.267523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4a0466bd-f68d-4efe-86ae-2b91dac3436c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:03.270910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4a0466bd-f68d-4efe-86ae-2b91dac3436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.281499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 862/1741] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:03.282945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.283487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.283882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.291324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 880/1742] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:42:03.292329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:03.298318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.309772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] MFA Rules not processed for user `d5e6d9fad06449c1b43254df442d224f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:03.311759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.322086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 863/1743] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:03.323445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:03.338333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1070a-0802-445b-a69e-043a1b8ac9c4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:03.344603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.345873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.345873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.347886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.349899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:03.350618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:03.355218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:03.371429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 864/1744] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:03.373757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:03.376068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:03.388502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:03.418538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:03.445476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515874e-d648-4dce-b05e-3c69d63ec386 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:03.459144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.459828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.460118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.460399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.482387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:03.484544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:03.518598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 881/1745] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:03.518598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:03.556156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9506f-29da-4873-af0f-aec84c61884d service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:03.576345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.578806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.580994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.581445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.583685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:03.595534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:03.697351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 882/1746] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:03 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:03.700752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:03.710381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f68bca-8a1b-473d-b31a-aba2df136af7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:03.919090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.919980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.920405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.920785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.958449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55959dcd-4f47-4790-84c2-9fa4d26fe245 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:03.980882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:03.981713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:03.982109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:03.982782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:03.983555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:03.985704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.054797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 883/1747] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:03 2022] POST /identity/v3/users => generated 294 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:04.056025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.080640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b302-85c3-44dd-b9b0-8507496dfe27 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:04.094807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:04.095642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.095985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.096203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.098524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.100271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.101871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.104465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.132204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 884/1748] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:04.142453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.151186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 865/1749] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:04.164386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f31ba-77a2-4ffc-b813-7d7bcd3c57b0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:04.174428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:04.175065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.175410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.175665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.176498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.178282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf187a-47ff-4a81-b169-4225915fc27a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.215693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.223626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 885/1750] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:04.237132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573375d4-46ae-4e9d-a1fb-6fa0eda99b50 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:04.274114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:04.279113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.281691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.282175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] PATH_INFO: `/v3/projects/777a322a541846b2b276fa60930d5ddf/users/8de49d2e503045cba20b987f9a1f1547/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.337226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777a322a541846b2b276fa60930d5ddf, user_id=8de49d2e503045cba20b987f9a1f1547, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.342566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.398141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 866/1751] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:04 2022] PUT /identity/v3/projects/777a322a541846b2b276fa60930d5ddf/users/8de49d2e503045cba20b987f9a1f1547/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:04.412573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.413101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.414707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.477252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.481158 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:04.545775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] MFA Rules not processed for user `8de49d2e503045cba20b987f9a1f1547`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:04.547586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:04.549403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:04.638312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cd419-4209-4ea4-9d9e-a50c645df9b2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:04.652181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:04.653222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.653545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.653824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] PATH_INFO: `/v3/projects/a21dba8e7c77448e8c60b133231597c3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.658472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21dba8e7c77448e8c60b133231597c3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.661786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:04.726802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 886/1752] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:04.727811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.734471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:04.793932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c203d-699a-4f65-8114-5a9533bc7bc4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:04.798181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21dba8e7c77448e8c60b133231597c3', 'request_id': 'req-644fb666-6e3a-4fca-948d-a218d9eced82'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:04.805835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:04.823705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21dba8e7c77448e8c60b133231597c3', 'request_id': 'req-644fb666-6e3a-4fca-948d-a218d9eced82'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:04.844794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21dba8e7c77448e8c60b133231597c3', 'request_id': 'req-644fb666-6e3a-4fca-948d-a218d9eced82'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:04.881608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] The token cache is being invalidate because project a21dba8e7c77448e8c60b133231597c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:04.883357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:04.884732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:04.888780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 867/1753] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:04 2022] DELETE /identity/v3/projects/a21dba8e7c77448e8c60b133231597c3 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:04.890168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:04.897376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:04.962046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:04.964704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:04.964704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:04.966599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] PATH_INFO: `/v3/users/229ef186ef0b437e8c9136563ecea01a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:04.969762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=229ef186ef0b437e8c9136563ecea01a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:04.978009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:05.073341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229ef186ef0b437e8c9136563ecea01a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.085761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fb666-6e3a-4fca-948d-a218d9eced82 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:05.101779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229ef186ef0b437e8c9136563ecea01a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.104460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229ef186ef0b437e8c9136563ecea01a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.108641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:05.109446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.109721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.110232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:05.111345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:05.114608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:05.130714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229ef186ef0b437e8c9136563ecea01a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.159543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229ef186ef0b437e8c9136563ecea01a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.174888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 868/1754] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:04 2022] POST /identity/v3/projects => generated 385 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:05.179225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:05.181640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:05.189254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 887/1755] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:04 2022] DELETE /identity/v3/users/229ef186ef0b437e8c9136563ecea01a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:05.190425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:05.195982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:05.259104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:05.346459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44efb5e-b188-41df-a618-5d5fab58b6fc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:05.406967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e13e3-3890-44f0-8af0-93708b1fa7ca admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:05.424981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:05.425745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.426037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.426320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:05.427030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:05.429015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f06180-346a-48f0-9896-4c91349e20c3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:05.457331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:05.458534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.459033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.459497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] PATH_INFO: `/v3/users/77b0215d958844b3b241482de0d1bf63` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:05.464703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b0215d958844b3b241482de0d1bf63)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:05.468087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:05.554818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 888/1756] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:05 2022] POST /identity/v3/users => generated 299 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:05.569948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a07ff-5ea3-4f7b-b713-66480618d956 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.575463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a07ff-5ea3-4f7b-b713-66480618d956 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.577006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a07ff-5ea3-4f7b-b713-66480618d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:05.596667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b0215d958844b3b241482de0d1bf63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.615775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b0215d958844b3b241482de0d1bf63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.643267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b0215d958844b3b241482de0d1bf63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.643267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b0215d958844b3b241482de0d1bf63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.643267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a268d7dd-9490-4fa2-91f7-aad5353baad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b0215d958844b3b241482de0d1bf63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:05.681504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 869/1757] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:05 2022] DELETE /identity/v3/users/77b0215d958844b3b241482de0d1bf63 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:05.686291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.686291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.686586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:05.698225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 870/1758] 10.222.0.65 () {54 vars in 912 bytes} [Fri Jun 24 09:42:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:42:05.698225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:05.710244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:05.710766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ec5a07ff-5ea3-4f7b-b713-66480618d956 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:05.712855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec5a07ff-5ea3-4f7b-b713-66480618d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:05.759635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:05.952472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 889/1759] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:05.970968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:05.971795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:05.972297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.053107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6072b-0c4c-4090-92ed-6e91dc1bd0cf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:06.054228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:06.055905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:06.065438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:06.066189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:06.066479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:06.067947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.071350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:06.077297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1786501-1a62-4845-b496-b07aae8a7070 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:06.092612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:06.114574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 871/1760] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:06.126692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:06.127015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:06.130262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.271713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] MFA Rules not processed for user `544cf118cc9e4aed8e7ec7e9dfe023bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:06.277211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:06.317177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 890/1761] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:42:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:06.320190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:06.326098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:06.407936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac251c3c-9e8e-4803-a192-34a17b09cd06 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:06.424767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:06.424767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:06.424767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:06.424767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.441306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:06.443268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:06.458268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 891/1762] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:06.459149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:06.473724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee190d9c-6f29-4dd2-91a9-778a0d101448 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:06.482851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:06.483756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:06.486565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:06.486941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] PATH_INFO: `/v3/projects/472df03721c94554ae71c6afd5379a16/users/83428c63b1ca4c388e227c344aeb2a90/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.523219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472df03721c94554ae71c6afd5379a16, user_id=83428c63b1ca4c388e227c344aeb2a90, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:06.526686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:06.555768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 892/1763] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:06 2022] PUT /identity/v3/projects/472df03721c94554ae71c6afd5379a16/users/83428c63b1ca4c388e227c344aeb2a90/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:06.558658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:06.564226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:06.779168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96cf7-556c-47e2-a563-7a0bab6d5a11 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:06.791378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:06.799780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:06.800196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:06.800490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:06.814770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 872/1764] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 699 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:06.815734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:06.837009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:06.849944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:06.882066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442afe1-2772-43a1-b838-bdf132070cc2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:06.882803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 893/1765] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:07.094685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:07.095733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.096188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.096730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.097796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:07.100471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c99dc-03a6-4c6d-b90e-711dd3548417 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:07.115711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 873/1766] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:07.119764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:07.134993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2209-8db7-4907-bfb0-d7b276a9103e service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:07.141300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:07.142567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.143238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.143733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] PATH_INFO: `/v3/projects/472df03721c94554ae71c6afd5379a16/users/83428c63b1ca4c388e227c344aeb2a90/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.146220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472df03721c94554ae71c6afd5379a16, user_id=83428c63b1ca4c388e227c344aeb2a90, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:07.149435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:07.164821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 894/1767] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:07 2022] PUT /identity/v3/projects/472df03721c94554ae71c6afd5379a16/users/83428c63b1ca4c388e227c344aeb2a90/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:07.181083 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.181602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.181992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.247800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] MFA Rules not processed for user `83428c63b1ca4c388e227c344aeb2a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:07.250609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:07.469589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 874/1768] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:07.490132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:07.493040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:07.564915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:07.657268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671a6f5-4e79-427d-a865-814c043721f1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:07.666401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:07.667388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.667790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.668219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.713539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:07.714537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:07.715672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:07.716162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae738f-cb8a-47c2-8149-229576514920 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:07.790730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 895/1769] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:07.798938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:07.829828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.830548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.832255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.935360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] MFA Rules not processed for user `fd136fcda9a04a16b258301a86bde8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:07.936903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:07.944516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d75e5-2ec8-4e76-97c1-eefa7b2bb329 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:07.962538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:07.962538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:07.962538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:07.962538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] PATH_INFO: `/v3/projects/421819eead3a48ea8ee00f8d778cf602` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:07.964023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421819eead3a48ea8ee00f8d778cf602)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:07.966762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:08.114069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421819eead3a48ea8ee00f8d778cf602', 'request_id': 'req-74ba0472-12ff-4b27-b579-01db94512583'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.133632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421819eead3a48ea8ee00f8d778cf602', 'request_id': 'req-74ba0472-12ff-4b27-b579-01db94512583'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.146257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421819eead3a48ea8ee00f8d778cf602', 'request_id': 'req-74ba0472-12ff-4b27-b579-01db94512583'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.172472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] The token cache is being invalidate because project 421819eead3a48ea8ee00f8d778cf602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:08.172769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.178899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.181177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 875/1770] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:07 2022] DELETE /identity/v3/projects/421819eead3a48ea8ee00f8d778cf602 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:08.182313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:08.184684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:08.224564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 896/1771] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:08.240518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:08.276564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51fdc-4598-4421-89a8-89ce161ceb8c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:08.500221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def073-6bec-443e-a223-983110560044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:08.501220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def073-6bec-443e-a223-983110560044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:08.501564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def073-6bec-443e-a223-983110560044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:08.501924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def073-6bec-443e-a223-983110560044 admin admin] PATH_INFO: `/v3/users/2a450a3ee8c040358143867fcef44588` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:08.502073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba0472-12ff-4b27-b579-01db94512583 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:08.507243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def073-6bec-443e-a223-983110560044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a450a3ee8c040358143867fcef44588)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:08.509962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def073-6bec-443e-a223-983110560044 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:08.526797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:08.527832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:08.528100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:08.530905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] PATH_INFO: `/v3/users/8de49d2e503045cba20b987f9a1f1547` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:08.535339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de49d2e503045cba20b987f9a1f1547)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:08.540309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:08.623838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a450a3ee8c040358143867fcef44588'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.651970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:08.661651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a450a3ee8c040358143867fcef44588'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.662565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a450a3ee8c040358143867fcef44588'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.681877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a450a3ee8c040358143867fcef44588'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.707478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a450a3ee8c040358143867fcef44588'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.718894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 897/1772] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:08 2022] DELETE /identity/v3/users/2a450a3ee8c040358143867fcef44588 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:08.720230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:08.724420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:08.739892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de49d2e503045cba20b987f9a1f1547'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.768149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:08.783679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de49d2e503045cba20b987f9a1f1547'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.812486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de49d2e503045cba20b987f9a1f1547'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.816561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de49d2e503045cba20b987f9a1f1547'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.817449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a8ebfd5f-04ce-4952-995c-6606ff47aa34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de49d2e503045cba20b987f9a1f1547'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:08.876524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 876/1773] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:08 2022] DELETE /identity/v3/users/8de49d2e503045cba20b987f9a1f1547 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:08.878161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc44b3-2a30-4107-a152-30e32046feaf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:08.883039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc44b3-2a30-4107-a152-30e32046feaf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:08.883603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc44b3-2a30-4107-a152-30e32046feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:08.989319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-46bc44b3-2a30-4107-a152-30e32046feaf None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:08.992255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46bc44b3-2a30-4107-a152-30e32046feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.075760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def073-6bec-443e-a223-983110560044 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:09.205039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 877/1774] 10.222.0.47 () {58 vars in 1015 bytes} [Fri Jun 24 09:42:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:09.211321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:09.211771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:09.217255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:09.295460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:09.297356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.325588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:09.326283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:09.326645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:09.329529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:09.336189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.520483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:09.522008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:09.572783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 898/1775] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 853 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:09.575768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:09.587911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.590269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 878/1776] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:09.593870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:09.599177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.637737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:09.654655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099daaba-605e-4f13-8217-6b551f2ee80c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:09.672958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:09.674161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:09.678645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:09.679476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:09.686033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8ff05-0d41-43da-a354-05430a333938 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:09.711738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:09.712784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:09.713053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:09.713343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] PATH_INFO: `/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:09.713528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:09.714748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f94830bf8934c1da29a16d2a7ad21b9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:09.720229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:09.721004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:09.756732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 879/1777] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:09.757481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:09.762974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:09.884711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7d6f3-5ee2-43dc-8811-a12bdb2078ff service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:09.946067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f94830bf8934c1da29a16d2a7ad21b9', 'request_id': 'req-0723f0ba-28e9-474b-bc46-3ba762ab43fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:09.968998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f94830bf8934c1da29a16d2a7ad21b9', 'request_id': 'req-0723f0ba-28e9-474b-bc46-3ba762ab43fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:09.989802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f94830bf8934c1da29a16d2a7ad21b9', 'request_id': 'req-0723f0ba-28e9-474b-bc46-3ba762ab43fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:10.036974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] The token cache is being invalidate because project 3f94830bf8934c1da29a16d2a7ad21b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:10.036974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:10.040247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:10.042429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 899/1778] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:09 2022] DELETE /identity/v3/projects/3f94830bf8934c1da29a16d2a7ad21b9 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:10.043045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:10.048030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.130672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:10.158152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:10.158909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.160122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.160419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.169730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.242515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723f0ba-28e9-474b-bc46-3ba762ab43fa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:10.250991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:10.252035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.252315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.252595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.257981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.384446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:10.393184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb122f49-fe7c-4fb5-ae0d-faf795599d79 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:10.456399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 880/1779] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:10.458017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47285-dd98-4875-b641-db2f197a7169 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.459159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:10.459584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47285-dd98-4875-b641-db2f197a7169 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.459971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47285-dd98-4875-b641-db2f197a7169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.475191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:10.508031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 900/1780] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:10.510634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:10.513675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.525362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-20a47285-dd98-4875-b641-db2f197a7169 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:10.530295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-20a47285-dd98-4875-b641-db2f197a7169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.546462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceb94e-dcd1-4c6b-82b3-2244ab9be3d7 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:10.561627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:10.562412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.562701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.563094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] PATH_INFO: `/v3/projects/98fcd55e45544fb2851eff516f27b25d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.576286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 881/1781] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:10.576786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fcd55e45544fb2851eff516f27b25d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:10.576786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:10.578738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.579168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.579549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.645521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:10.652748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:10.693443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 882/1782] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:10.696115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:10.726865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51c1dd-56d6-4be7-89e5-07095a668092 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:10.774840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:10.775780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.776273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.778302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.779221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:10.781364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:10.789733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:10.846526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 883/1783] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:10.847656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:10.909741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a61d80-495a-4cfd-b5fa-fbe8a09d35f0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:10.968739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:10.971605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:10.971956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:10.972266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:10.976286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:10.986641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:11.150762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 884/1784] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:10 2022] POST /identity/v3/projects => generated 369 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:11.157030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:11.245666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:11.356082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fcd55e45544fb2851eff516f27b25d', 'request_id': 'req-a79bf0b3-cf08-48cb-93f4-e3867853f937'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:11.370046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fcd55e45544fb2851eff516f27b25d', 'request_id': 'req-a79bf0b3-cf08-48cb-93f4-e3867853f937'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:11.386692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fcd55e45544fb2851eff516f27b25d', 'request_id': 'req-a79bf0b3-cf08-48cb-93f4-e3867853f937'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:11.427911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] The token cache is being invalidate because project 98fcd55e45544fb2851eff516f27b25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:11.429371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:11.435465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a79bf0b3-cf08-48cb-93f4-e3867853f937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:11.441190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 901/1785] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:10 2022] DELETE /identity/v3/projects/98fcd55e45544fb2851eff516f27b25d => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:11.446772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1804c9e-03b7-4fde-9ac3-ededb27930c4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:11.446977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1804c9e-03b7-4fde-9ac3-ededb27930c4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:11.447414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1804c9e-03b7-4fde-9ac3-ededb27930c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:11.512619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a1804c9e-03b7-4fde-9ac3-ededb27930c4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:11.517661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a1804c9e-03b7-4fde-9ac3-ededb27930c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:11.642146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f97bc-5d23-405d-b629-dbb578765133 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:11.735365 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 902/1786] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:11.737028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:11.737330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:11.737926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:11.840299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] MFA Rules not processed for user `d5e6d9fad06449c1b43254df442d224f`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:11.841547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:11.938536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:11.961770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:11.962386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:11.962684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:11.962961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:11.963789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:11.966567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc391ac1-4add-4fa2-8e13-c947a5797b9a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.023614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 885/1787] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:11 2022] POST /identity/v3/users => generated 285 bytes in 872 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:12.037057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.037463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.037755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.102883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:12.113934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:12.138847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 903/1788] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:12.139137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 886/1789] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:12.143695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.145332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.154896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:12.177089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01852fe-8844-4feb-9098-cd09ed38dd29 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.197686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.198382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.199266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.199266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.199954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:12.200243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.202049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.219960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b305a4-b367-4f0e-8b61-5967a1230be7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.234253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 887/1790] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:12.235253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.240504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.244172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.244486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.244771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.245554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.257220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.258026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fd8eb-2777-41c3-b309-93963d61beab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.268153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.268857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.269146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.269432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.272934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.280515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.283036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 904/1791] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:12.287357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.317926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6359b-eacb-4395-90de-537b5ac93f54 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.330516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.331351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.331622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.331983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.332731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.335345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.336303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 888/1792] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:12 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:12.340978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.371221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 905/1793] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:12.375537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.402711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f7104-a016-4556-9ad3-856de655cbb0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.546418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b5b4e-faf1-4494-a2ff-37c3c1163f75 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.567639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:12.568375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.569305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.569778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.570561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.572796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.580711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.638306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.643800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.644093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.644379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] PATH_INFO: `/v3/projects/bdf9b50268aa4f58bbc2b766d5f0bf78/users/c3bfcc0dac714c019e2bfd3eb0d6bffe/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.661913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 889/1794] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:12 2022] POST /identity/v3/users => generated 290 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:12.662938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.709863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a978bc-4230-4e5f-acbd-2937af42a84a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.714834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf9b50268aa4f58bbc2b766d5f0bf78, user_id=c3bfcc0dac714c019e2bfd3eb0d6bffe, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.722281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cb974-63f8-42de-ad52-4c2161f6092e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.735551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:12.736348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.736671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.737051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.737945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:12.739801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:12.781022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 906/1795] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:12 2022] PUT /identity/v3/projects/bdf9b50268aa4f58bbc2b766d5f0bf78/users/c3bfcc0dac714c019e2bfd3eb0d6bffe/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:12.782739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:12.783135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:12.783444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:12.793864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 890/1796] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:12.802763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:12.823428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] MFA Rules not processed for user `c3bfcc0dac714c019e2bfd3eb0d6bffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:12.826740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:12.858320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88773b84-fa0d-4736-ac2e-d507a13f07a2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:12.981222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 907/1797] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:12.982712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.157169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.157900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.160212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.160482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.161253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.170641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98945-375d-445f-a87c-79913af5ed0f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.176145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.195027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.198659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.200048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.201616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.207030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 891/1798] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:13.208608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.209794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.221462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.245673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090377eb-d011-4a38-b613-e97c86a675a3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.256750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.257742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.258035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.258340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] PATH_INFO: `/v3/projects/3ca69ec711704416a2de527628e6231c/users/bd6343e9405a49eb9b249232af329cb8/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.288800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:13.297187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca69ec711704416a2de527628e6231c, user_id=bd6343e9405a49eb9b249232af329cb8, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.304556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac2703-1336-439f-8feb-aa464a049039 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.314582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 908/1799] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:12 2022] POST /identity/v3/projects => generated 369 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:13.318863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.367153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 892/1800] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:13 2022] PUT /identity/v3/projects/3ca69ec711704416a2de527628e6231c/users/bd6343e9405a49eb9b249232af329cb8/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:13.377988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.378297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.378585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.470389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] MFA Rules not processed for user `bd6343e9405a49eb9b249232af329cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:13.474615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:13.488241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:13.517839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e2343-b538-48fe-bd9f-9afebffec974 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.648598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.652654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.653583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.654177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.655261 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.661762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.708735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 909/1801] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:13 2022] POST /identity/v3/users => generated 291 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:13.712817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.734904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9d3b-9972-4c68-86f9-a7b8a5a7e1a6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.757523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 893/1802] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:13.758914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.766930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.768599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.769098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.769423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.775595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.777334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.786626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f96c9-b5f8-4381-88a8-38d9c169106f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.795910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:13.796703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:13.797003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:13.797363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:13.798115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:13.802429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:13.804929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 910/1803] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:13.806466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:13.820813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:13.900716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4eca-fb19-4755-b837-d631e56bfaa9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:13.907178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 894/1804] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:13 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:13.908371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.055308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeca6c2-88a0-4aa0-872c-76480170f333 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:14.095193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.096804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:14.097305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:14.097914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] PATH_INFO: `/v3/projects/101f7f0141924ccb8ea81eac54751599/users/303b81a5986d4255ac00445099970a7c/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:14.147657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=101f7f0141924ccb8ea81eac54751599, user_id=303b81a5986d4255ac00445099970a7c, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:14.165810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:14.253167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 895/1805] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:13 2022] PUT /identity/v3/projects/101f7f0141924ccb8ea81eac54751599/users/303b81a5986d4255ac00445099970a7c/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:14.259122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.279028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.279974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:14.280251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:14.280530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] PATH_INFO: `/v3/projects/777a322a541846b2b276fa60930d5ddf` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:14.285597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777a322a541846b2b276fa60930d5ddf)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:14.291540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:14.495987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777a322a541846b2b276fa60930d5ddf', 'request_id': 'req-9d90961b-f946-4012-b868-7e76c8692614'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:14.517753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:14.519882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777a322a541846b2b276fa60930d5ddf', 'request_id': 'req-9d90961b-f946-4012-b868-7e76c8692614'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:14.561519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777a322a541846b2b276fa60930d5ddf', 'request_id': 'req-9d90961b-f946-4012-b868-7e76c8692614'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:14.579259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:14.587539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4e31f-615a-4fe2-bb50-51b2aa32d19f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:14.592120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] The token cache is being invalidate because project 777a322a541846b2b276fa60930d5ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:14.595249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:14.597787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:14.611536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 911/1806] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:13 2022] DELETE /identity/v3/projects/777a322a541846b2b276fa60930d5ddf => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:14.612551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.623748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:14.844827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90961b-f946-4012-b868-7e76c8692614 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:14.851361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.852193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:14.852474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:14.852767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:14.855547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:14.863320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:14.878104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.878922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:14.879290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:14.882464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:14.887035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:14.895757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:14.916614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 896/1807] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:14 2022] POST /identity/v3/users => generated 297 bytes in 663 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:14.922010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.929589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:14.962866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 912/1808] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:14.963989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:14.975119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:14.976286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9359fa8-b17b-4f71-a893-7591a41cdcfc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:14.998789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:14.999738 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.000020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.000298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] PATH_INFO: `/v3/users/83428c63b1ca4c388e227c344aeb2a90` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.012792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83428c63b1ca4c388e227c344aeb2a90)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.018934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.020807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f2e10-fe1c-4dcc-91c0-ff6c4ea1e432 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.053173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:15.055054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.057330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.058431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.061499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.067498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df5a34-a8aa-41bf-b166-31a322cdc3e6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.129570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 913/1809] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:15.135799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f11abe-abda-40b3-91f6-303e1e65ea1e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.138736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f11abe-abda-40b3-91f6-303e1e65ea1e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.139111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f11abe-abda-40b3-91f6-303e1e65ea1e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.145519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 914/1810] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:42:15 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:42:15.147503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f11abe-abda-40b3-91f6-303e1e65ea1e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:15.160577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83428c63b1ca4c388e227c344aeb2a90'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:15.178484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83428c63b1ca4c388e227c344aeb2a90'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:15.203822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83428c63b1ca4c388e227c344aeb2a90'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:15.204474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83428c63b1ca4c388e227c344aeb2a90'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:15.205213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83428c63b1ca4c388e227c344aeb2a90'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:15.207015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f11abe-abda-40b3-91f6-303e1e65ea1e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.224354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:15.226530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.226820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.229645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] PATH_INFO: `/v3/projects/101f7f0141924ccb8ea81eac54751599/users/303b81a5986d4255ac00445099970a7c/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.237649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101f7f0141924ccb8ea81eac54751599, user_id=303b81a5986d4255ac00445099970a7c, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.243517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 897/1811] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:14 2022] DELETE /identity/v3/users/83428c63b1ca4c388e227c344aeb2a90 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:15.247253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:15.251659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:15.262550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d866ca-3e4b-47d7-81ce-140fe790c670 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.304439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 915/1812] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:15 2022] PUT /identity/v3/projects/101f7f0141924ccb8ea81eac54751599/users/303b81a5986d4255ac00445099970a7c/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:15.326351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70d98-e258-46bf-a3b8-9f2907becf0b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.326661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70d98-e258-46bf-a3b8-9f2907becf0b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.326960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70d98-e258-46bf-a3b8-9f2907becf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.348042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:15.368727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff382c5-6dd0-4738-8778-c07a1d3ce822 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.442957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d2f70d98-e258-46bf-a3b8-9f2907becf0b None None] MFA Rules not processed for user `303b81a5986d4255ac00445099970a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:15.443933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d2f70d98-e258-46bf-a3b8-9f2907becf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:15.457784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:15.461114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.462487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.463758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] PATH_INFO: `/v3/projects/222338cb9d174e3ea8edc539a49a76dd/users/bca16595bb5d4912b4bd2cbb7ea4e471/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.507709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222338cb9d174e3ea8edc539a49a76dd, user_id=bca16595bb5d4912b4bd2cbb7ea4e471, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.515905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.551413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 898/1813] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:15 2022] PUT /identity/v3/projects/222338cb9d174e3ea8edc539a49a76dd/users/bca16595bb5d4912b4bd2cbb7ea4e471/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:15.555504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:15.674810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 916/1814] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:15.679476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.679844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.680191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.730480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f124dcc-d2c0-4aea-bd5a-8391df9a2343 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.740644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:15.741335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.741620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.741902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.742688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.747057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.752157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:15.753540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:15.773754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 899/1815] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:15.775692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:15.778908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:15.838741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:15.863188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 917/1816] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:42:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:15.863976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:15.876232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60000839-4eb4-456b-a75f-716ad9d72d52 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.886093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:15.887887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.887887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.888129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] PATH_INFO: `/v3/projects/222338cb9d174e3ea8edc539a49a76dd/users/bca16595bb5d4912b4bd2cbb7ea4e471/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.890964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=222338cb9d174e3ea8edc539a49a76dd, user_id=bca16595bb5d4912b4bd2cbb7ea4e471, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:15.893722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c9f82-8400-4c84-80e9-1e2ac230aeae admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:15.909595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 918/1817] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:15 2022] PUT /identity/v3/projects/222338cb9d174e3ea8edc539a49a76dd/users/bca16595bb5d4912b4bd2cbb7ea4e471/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:15.923399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f646-acbc-4120-9cc6-37adf6498bc8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:15.923837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f646-acbc-4120-9cc6-37adf6498bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:15.924194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f646-acbc-4120-9cc6-37adf6498bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:15.943919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569383-1697-4314-a1ac-db77a97fff21 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:15.980064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b290f646-acbc-4120-9cc6-37adf6498bc8 None None] MFA Rules not processed for user `bca16595bb5d4912b4bd2cbb7ea4e471`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:15.982016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b290f646-acbc-4120-9cc6-37adf6498bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:16.022700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:16.049924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:16.050779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.051195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.051595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.171315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 919/1818] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:16.171315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.171315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.171315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.183068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:16.199178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:16.211983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] MFA Rules not processed for user `544cf118cc9e4aed8e7ec7e9dfe023bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:16.214424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:16.217132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 900/1819] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:16.218585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:16.220166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:16.353084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 920/1820] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:16.356857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:16.359606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:16.374931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885c2f7-8313-4ee3-b7c3-61c3562bba9d service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:16.382592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:16.383833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.384431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.384823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.392612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ff4ac-8db8-4607-9fc7-7c976727f1dd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:16.397429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:16.398121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.398410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.398694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.402529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:16.405055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:16.422212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:16.423777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1f11b-5cb4-4290-a24b-3fe4b32974eb service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:16.445584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 901/1821] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:16.447738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 921/1822] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:16.460297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:16.462673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:16.643447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e91ea-70e0-44d6-87ba-0cbc7fd5a6a1 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:16.650798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:16.652024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.653017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.653376 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] PATH_INFO: `/v3/projects/472df03721c94554ae71c6afd5379a16` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.655167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] RBAC: Authorizing `identity:delete_project(project_id=472df03721c94554ae71c6afd5379a16)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:16.658662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:16.675657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:16.739890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472df03721c94554ae71c6afd5379a16', 'request_id': 'req-b77ead3e-f175-4e1f-9136-38e240c5464f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:16.787833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472df03721c94554ae71c6afd5379a16', 'request_id': 'req-b77ead3e-f175-4e1f-9136-38e240c5464f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:16.799633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472df03721c94554ae71c6afd5379a16', 'request_id': 'req-b77ead3e-f175-4e1f-9136-38e240c5464f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:16.816712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] The token cache is being invalidate because project 472df03721c94554ae71c6afd5379a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:16.817204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:16.817956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b77ead3e-f175-4e1f-9136-38e240c5464f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:16.827391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:16.828206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:16.828569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:16.830932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 902/1823] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:16 2022] DELETE /identity/v3/projects/472df03721c94554ae71c6afd5379a16 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:16.872308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:16.873589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:17.012832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:17.044351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 922/1824] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:17.052668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.053151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.053312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.102707 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:17.104053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:17.135832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 903/1825] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:17.140210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.155977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477404b8-8dee-43e7-a093-eb50b5cee978 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.172702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.173438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.173723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.173997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.177143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:17.177468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.181464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.221067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 923/1826] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:17.227110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.257667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b1b1f-03b5-4358-afd2-79c12e60b33e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.275481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.276427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.276962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.277155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.278800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.284472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.387624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.392469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:17.400159 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 904/1827] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:17 2022] POST /identity/v3/projects => generated 401 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:17.402716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.434473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e101a-2b94-4eb6-a435-09cfcef8d0e4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.620160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114bbe1-cb65-41d7-b45c-4b90472c146a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.628981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.630160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.633929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.634410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.635512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.641157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.662857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.668914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.670671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.670969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] PATH_INFO: `/v3/users/bd6343e9405a49eb9b249232af329cb8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.678201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6343e9405a49eb9b249232af329cb8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.679579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 905/1828] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:17 2022] POST /identity/v3/users => generated 301 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:17.681912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.683463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.724504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4e162-84ea-4842-b3fa-a178e8811889 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.739434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.741104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.741409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.741693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.742836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.752602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.787474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 906/1829] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:17.793883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.813831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb62c7f-8364-45a1-ab2b-54ed06119769 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.814596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6343e9405a49eb9b249232af329cb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:17.822148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:17.823362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.823654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.823974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.825303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:17.828437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:17.846267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6343e9405a49eb9b249232af329cb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:17.847063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6343e9405a49eb9b249232af329cb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:17.857553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 907/1830] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:17.862164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:17.868736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6343e9405a49eb9b249232af329cb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:17.881590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-60dd0ff0-fe8f-4b5b-9ec9-ff95971a1c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6343e9405a49eb9b249232af329cb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:17.890575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccee3e-2aaf-464d-84fb-f43cb0cefb7a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:17.891784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 924/1831] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:17 2022] DELETE /identity/v3/users/bd6343e9405a49eb9b249232af329cb8 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:17.893253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:17.893684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:17.893971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:17.917738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:17.921260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:18.077195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:18.083503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:18.125026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 925/1832] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:18.126353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:18.134361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:18.156953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:18.158023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:18.159523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:18.159891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] PATH_INFO: `/v3/projects/bd2861392d3c409fa30edd4f2ebbc37d/users/d77d6846b6df42289beba7594bfa06e1/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:18.189453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2861392d3c409fa30edd4f2ebbc37d, user_id=d77d6846b6df42289beba7594bfa06e1, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:18.193549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48695bba-4f35-438e-bc86-184de7c589d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:18.246526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 908/1833] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:17 2022] PUT /identity/v3/projects/bd2861392d3c409fa30edd4f2ebbc37d/users/d77d6846b6df42289beba7594bfa06e1/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:18.263203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:18.271332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:18.273574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:18.341816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83c6ad-0325-4e3b-9313-0f59a39848e0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:18.343205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] MFA Rules not processed for user `d77d6846b6df42289beba7594bfa06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:18.346774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:18.595534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:18.600709 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:18.605844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:18.606428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:18.647789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 909/1834] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:18.651451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:18.662722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:18.781526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:18.783025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:18.810879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d712b-b600-4d5a-b91c-72e09b20c2a7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:18.812434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 926/1835] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:18.825802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:18.826797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:18.827364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:18.828095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:18.829017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:18.831889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:18.872477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 910/1836] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:18 2022] POST /identity/v3/projects => generated 401 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:18.875994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:18.890131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2c131-47dc-4c0e-8942-51395eedfd23 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.057151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.057866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.058216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.058428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.059167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.061509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.075396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:19.110892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 927/1837] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:18 2022] POST /identity/v3/users => generated 307 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:19.112214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.127998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ffbb2-5f18-4781-bab6-d8d956513450 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.149788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.151383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.151383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.151383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.152074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.156136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.192789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 911/1838] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:19.194209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.208878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c5a47-8433-42cc-9ef5-cb89057f2fab admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.223256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.226614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.226989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.228670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] PATH_INFO: `/v3/projects/9fcae4f6fb9142d78ceb270a0476694f/users/4a60a304ff894cb38eaa6fa55a2d464e/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.272349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcae4f6fb9142d78ceb270a0476694f, user_id=4a60a304ff894cb38eaa6fa55a2d464e, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.279457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.342424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.351100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 928/1839] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:19 2022] PUT /identity/v3/projects/9fcae4f6fb9142d78ceb270a0476694f/users/4a60a304ff894cb38eaa6fa55a2d464e/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:19.360527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff03cf-0b3b-4060-a1d2-55edd99e1334 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.409058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:19.468059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.468754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.469039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.469411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.470157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.472195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.485096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 912/1840] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:19.489426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.507175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e951-06cc-4d29-96f8-1f16e1d7f897 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.516431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.518798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:19.529864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.533140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.533652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.534595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] PATH_INFO: `/v3/projects/9fcae4f6fb9142d78ceb270a0476694f/users/4a60a304ff894cb38eaa6fa55a2d464e/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.542889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcae4f6fb9142d78ceb270a0476694f, user_id=4a60a304ff894cb38eaa6fa55a2d464e, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.556758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38ede5-6902-4ac0-b100-cef88ede219e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.558392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d877c-27c0-4c55-a960-6d7662f81443 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.568167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.569645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.570084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.570599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] PATH_INFO: `/v3/projects/3ca69ec711704416a2de527628e6231c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.572442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca69ec711704416a2de527628e6231c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.575644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 929/1841] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:19 2022] PUT /identity/v3/projects/9fcae4f6fb9142d78ceb270a0476694f/users/4a60a304ff894cb38eaa6fa55a2d464e/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:19.577922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.587443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36840-2319-47cb-822a-ee0d728123e2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.587878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36840-2319-47cb-822a-ee0d728123e2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.588304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36840-2319-47cb-822a-ee0d728123e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.637971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d5a36840-2319-47cb-822a-ee0d728123e2 None None] MFA Rules not processed for user `4a60a304ff894cb38eaa6fa55a2d464e`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:19.639385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d5a36840-2319-47cb-822a-ee0d728123e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:19.683823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca69ec711704416a2de527628e6231c', 'request_id': 'req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.714511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca69ec711704416a2de527628e6231c', 'request_id': 'req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.724965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca69ec711704416a2de527628e6231c', 'request_id': 'req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.739241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] The token cache is being invalidate because project 3ca69ec711704416a2de527628e6231c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:19.739347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.739989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.751561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 913/1842] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:19 2022] DELETE /identity/v3/projects/3ca69ec711704416a2de527628e6231c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:19.752464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.753881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:19.799580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 930/1843] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:19.799580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.799841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.800122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.824767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] MFA Rules not processed for user `fd136fcda9a04a16b258301a86bde8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:19.826167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:19.861318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f8ea-4317-4103-9ca9-1a7077e84f9d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.869136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.870054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.870334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.870615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] PATH_INFO: `/v3/users/c3bfcc0dac714c019e2bfd3eb0d6bffe` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.871946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3bfcc0dac714c019e2bfd3eb0d6bffe)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.880471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.902027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 931/1844] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:19.910485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:19.912098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:19.937488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c461c7e-0e72-49ba-82d8-67be863af976 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:19.948841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:19.949770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:19.950056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:19.950375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] PATH_INFO: `/v3/users/bca16595bb5d4912b4bd2cbb7ea4e471` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:19.951660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca16595bb5d4912b4bd2cbb7ea4e471)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:19.954160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:19.973387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:19.981455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3bfcc0dac714c019e2bfd3eb0d6bffe'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:19.995924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3bfcc0dac714c019e2bfd3eb0d6bffe'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.010891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3bfcc0dac714c019e2bfd3eb0d6bffe'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.011629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3bfcc0dac714c019e2bfd3eb0d6bffe'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.012281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3bfcc0dac714c019e2bfd3eb0d6bffe'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.040128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:20.058214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 914/1845] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:19 2022] DELETE /identity/v3/users/c3bfcc0dac714c019e2bfd3eb0d6bffe => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:20.058859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca16595bb5d4912b4bd2cbb7ea4e471'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.066091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:20.067742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.085599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca16595bb5d4912b4bd2cbb7ea4e471'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.086471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca16595bb5d4912b4bd2cbb7ea4e471'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.114631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca16595bb5d4912b4bd2cbb7ea4e471'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.125807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6c54943e-c8d1-4a3c-b6f2-547f1f3a3253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca16595bb5d4912b4bd2cbb7ea4e471'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:20.144031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 932/1846] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:19 2022] DELETE /identity/v3/users/bca16595bb5d4912b4bd2cbb7ea4e471 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:20.144031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:20.144031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:20.144031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:20.192823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:20.199755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.214788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605cb13-a9ee-47e3-ae97-a484f56888f7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:20.303566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:20.303566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:20.303566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:20.303566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:20.310218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.349917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 933/1847] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:20.360362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:20.362456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.503942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682c99-c10e-4154-9a88-ccd149be2c37 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:20.509901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:20.511222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:20.511897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:20.512387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:20.526808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:20.530320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:20.537092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:20.541789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:20.556898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 934/1848] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:42:20.561293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 915/1849] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:20 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:20.694653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:20.696614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.712938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57f4e-64f1-4bc9-b3f5-7442398cfe79 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:20.716997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:20.717649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:20.718071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:20.718364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:20.720658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:20.796956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:20.805430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920859-c700-4f04-9274-16a65bd92a7d service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:20.813213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 935/1850] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:20.813836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:21.140157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:21.143152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:21.178709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb720e-d08f-4a0b-a506-1091a48a0ea6 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:21.203344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:21.207140 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:21.207697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:21.208069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] PATH_INFO: `/v3/projects/222338cb9d174e3ea8edc539a49a76dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:21.210207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222338cb9d174e3ea8edc539a49a76dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:21.214857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:21.359356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:21.415987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222338cb9d174e3ea8edc539a49a76dd', 'request_id': 'req-d6b5bd64-5b26-43a8-8961-83e1de286f39'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.433571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222338cb9d174e3ea8edc539a49a76dd', 'request_id': 'req-d6b5bd64-5b26-43a8-8961-83e1de286f39'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.448137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222338cb9d174e3ea8edc539a49a76dd', 'request_id': 'req-d6b5bd64-5b26-43a8-8961-83e1de286f39'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.497868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] The token cache is being invalidate because project 222338cb9d174e3ea8edc539a49a76dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:21.498155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.503290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:21.516035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 916/1851] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:21 2022] DELETE /identity/v3/projects/222338cb9d174e3ea8edc539a49a76dd => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:21.519027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da14e3-ca74-479a-b638-1779029cc587 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:21.520454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da14e3-ca74-479a-b638-1779029cc587 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:21.521279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da14e3-ca74-479a-b638-1779029cc587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:21.650136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-21da14e3-ca74-479a-b638-1779029cc587 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:21.651666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-21da14e3-ca74-479a-b638-1779029cc587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:21.691713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:21.700313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:21.882370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 936/1852] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:21.891533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:21.892012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:21.892431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:21.991370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:21.998438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.030623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 937/1853] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:22.042493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.052645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5bd64-5b26-43a8-8961-83e1de286f39 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.080351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.081357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.081719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.082023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] PATH_INFO: `/v3/projects/bdf9b50268aa4f58bbc2b766d5f0bf78` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.088538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef5ce-66f7-4d75-82c6-0717a51db798 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.093665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf9b50268aa4f58bbc2b766d5f0bf78)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.099519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.129473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.140333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.146676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.147222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.148725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:22.152523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.152523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.205260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 938/1854] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:22.208923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.236263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de3a18-fe34-4860-8038-92a1fa5481ea admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.286239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf9b50268aa4f58bbc2b766d5f0bf78', 'request_id': 'req-98590c4b-bf2a-4728-bc81-2a9dc45e969c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.320830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf9b50268aa4f58bbc2b766d5f0bf78', 'request_id': 'req-98590c4b-bf2a-4728-bc81-2a9dc45e969c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.354945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf9b50268aa4f58bbc2b766d5f0bf78', 'request_id': 'req-98590c4b-bf2a-4728-bc81-2a9dc45e969c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.383476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] The token cache is being invalidate because project bdf9b50268aa4f58bbc2b766d5f0bf78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:22.384423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.387284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.392865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 917/1855] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:21 2022] DELETE /identity/v3/projects/bdf9b50268aa4f58bbc2b766d5f0bf78 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:22.397027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.401181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.529333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:22.596738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.597532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.597869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.598060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.598757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.598988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98590c4b-bf2a-4728-bc81-2a9dc45e969c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.606155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.612955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.620254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.621610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.621935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] PATH_INFO: `/v3/users/303b81a5986d4255ac00445099970a7c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.625314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=303b81a5986d4255ac00445099970a7c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.632401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.682810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 939/1856] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:22 2022] POST /identity/v3/projects => generated 383 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:22.686963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.693737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.798191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303b81a5986d4255ac00445099970a7c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.816844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303b81a5986d4255ac00445099970a7c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.840049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bd1bd-9384-40e1-9028-77fe7b76479e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:22.858111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303b81a5986d4255ac00445099970a7c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.858963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303b81a5986d4255ac00445099970a7c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.860161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303b81a5986d4255ac00445099970a7c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:22.863826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:22.864920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:22.865297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:22.865572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:22.867627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:22.875535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:22.892027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:22.918008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 940/1857] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:22 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:22.919493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:22.921799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:22.932678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 918/1858] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:22 2022] DELETE /identity/v3/users/303b81a5986d4255ac00445099970a7c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:22.938641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89845c-a152-46d0-8daf-cce15df6b96a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:23.028823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:23.029596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:23.029933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:23.030280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:23.031032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:23.033342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:23.051451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 941/1859] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:23.052281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:23.068820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf305-d6f4-40ce-8cea-e997edcaaf6e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:23.095428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:23.098679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:23.098970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:23.099306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:23.106331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:23.112530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463e605-4f39-4857-92b3-00533311f431 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:23.148348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 919/1860] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:23.152852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:23.177163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc922807-bafa-48c5-9363-9fb831c64b5a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:23.198234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:23.199179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:23.199481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:23.199741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] PATH_INFO: `/v3/projects/8ffbc6eab1bc433ab46a95ea589c7dfe/users/3807b84b72a14926aa71c5b562b8e1ca/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:23.309314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffbc6eab1bc433ab46a95ea589c7dfe, user_id=3807b84b72a14926aa71c5b562b8e1ca, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:23.329343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:23.384186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 942/1861] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:23 2022] PUT /identity/v3/projects/8ffbc6eab1bc433ab46a95ea589c7dfe/users/3807b84b72a14926aa71c5b562b8e1ca/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:23.401142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:23.401423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:23.404295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:23.449001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] MFA Rules not processed for user `3807b84b72a14926aa71c5b562b8e1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:23.451664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:23.576031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 920/1862] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:23.635742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:23.638507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:23.890106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113702d3-ff62-40fd-9389-2f2a1c377a1c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:23.906297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:23.910025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:23.911542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:23.911874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:23.938710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:23.942470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3483141-dcf1-4eae-9dc3-0ce5e8f12810 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:23.965138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 943/1863] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:24.290871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:24.294505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:24.460450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:24.535713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13847f8b-4534-4423-ac7e-9c59ca0e9343 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:24.543896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:24.545632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:24.546043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:24.546748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] PATH_INFO: `/v3/projects/101f7f0141924ccb8ea81eac54751599` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:24.548402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101f7f0141924ccb8ea81eac54751599)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:24.551416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:24.626743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101f7f0141924ccb8ea81eac54751599', 'request_id': 'req-d93323c7-fcc5-4eaf-8529-b729101c47f2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:24.642214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101f7f0141924ccb8ea81eac54751599', 'request_id': 'req-d93323c7-fcc5-4eaf-8529-b729101c47f2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:24.655204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101f7f0141924ccb8ea81eac54751599', 'request_id': 'req-d93323c7-fcc5-4eaf-8529-b729101c47f2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:24.673766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] The token cache is being invalidate because project 101f7f0141924ccb8ea81eac54751599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:24.675106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:24.676666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d93323c7-fcc5-4eaf-8529-b729101c47f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:24.682322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 921/1864] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:24 2022] DELETE /identity/v3/projects/101f7f0141924ccb8ea81eac54751599 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:24.685432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:24.690227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:24.690921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:24.723672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:24.725751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:24.826308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 944/1865] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:24.834600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:24.834866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:24.837152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:24.876013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:24.877895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:24.899588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 922/1866] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:24.902618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:24.921078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c9d9f-ece9-40df-b114-85cb2a42ca9b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:24.935792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:24.937234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:24.939289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:24.940030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:24.941513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:24.943178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:24.948576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:24.967490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:24.983941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 945/1867] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:42:24.985174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.000795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d32c-973c-48fb-a85d-c6b2ff383617 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.015034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.019715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.020161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.034823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.035760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.039190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.088991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 923/1868] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:24 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:25.093150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.108606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f30ff-d4cf-404d-a5e5-3f39ee3f20ee admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.209731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.210058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.210339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.210617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.210617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.210617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.245760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 946/1869] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:25 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:25.251718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.268662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271fa9-7459-4232-9d50-712cc418444e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.286459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.288450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.288916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.291344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.292359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.299860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.338100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 924/1870] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:25.342396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.371109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f956-cdb8-416e-a89e-66cbd6a55ec2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.388961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.389724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.390012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.390304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.391019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.392724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc394a45-f526-4bff-b89b-774d1c4e8a16 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.412297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.418152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 947/1871] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:25.425000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2e198-8b64-4834-b950-e919d2e76aee admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.430315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.431519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.431978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.432434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] PATH_INFO: `/v3/projects/40230e4314e449339e244c3e69b0edce/users/8588cad0074c4b958813977a4c5366cd/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.454391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40230e4314e449339e244c3e69b0edce, user_id=8588cad0074c4b958813977a4c5366cd, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.459127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.474256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 925/1872] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:25 2022] PUT /identity/v3/projects/40230e4314e449339e244c3e69b0edce/users/8588cad0074c4b958813977a4c5366cd/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:25.486237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.486552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.486807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.519136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] MFA Rules not processed for user `8588cad0074c4b958813977a4c5366cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:25.521291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:25.628882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 948/1873] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:25.633183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.736171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776dd8c-678d-4e08-a83b-1d34e343863e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.741661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.742500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.743506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.743888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.744722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.747035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.763719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:25.781540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 926/1874] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:25 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:25.784922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.798847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043477c-7646-46f1-abfc-4ed5ffdb641a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.873943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.874670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.874958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.875282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.875985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.878138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:25.900459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:25.922970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 949/1875] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:25 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:25.926896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:25.963440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282f8bf-9088-4c64-8782-af8eb9ba3086 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:25.975980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:25.977249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:25.977736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:25.978302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:25.979269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:25.981529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.013783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 927/1876] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:26.018072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.039811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ec11-387c-40ca-a26c-04a9e5df76c8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.045560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.046969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.047303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.047609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] PATH_INFO: `/v3/projects/3b38f521c99a40e18c94515702f296d0/users/75e066765196408c810545f392588ded/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.073214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b38f521c99a40e18c94515702f296d0, user_id=75e066765196408c810545f392588ded, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.076505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.102350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 950/1877] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:26 2022] PUT /identity/v3/projects/3b38f521c99a40e18c94515702f296d0/users/75e066765196408c810545f392588ded/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:26.106181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.121557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756968e-5f21-4742-ba59-cd4f0e230528 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.209579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.210204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.210495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.210778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.211641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.213988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba33e6-e330-4247-befa-6894895c9a2d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.239815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 928/1878] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:26.241734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.258053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba081792-4132-4650-b966-35e00e7da1d4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.265481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.266682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.267476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.268011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] PATH_INFO: `/v3/projects/3b38f521c99a40e18c94515702f296d0/users/75e066765196408c810545f392588ded/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.270701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b38f521c99a40e18c94515702f296d0, user_id=75e066765196408c810545f392588ded, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.273945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.300597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 951/1879] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:26 2022] PUT /identity/v3/projects/3b38f521c99a40e18c94515702f296d0/users/75e066765196408c810545f392588ded/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:26.312276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.312744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.313046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.355696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] MFA Rules not processed for user `75e066765196408c810545f392588ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:26.358154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.392599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.393905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.485250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 929/1880] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:26.497289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.499987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.520905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2c1d9-1af5-448d-b82b-fe22c4fd5690 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.531790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.534702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.535132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.535485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] PATH_INFO: `/v3/users/d77d6846b6df42289beba7594bfa06e1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.538397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77d6846b6df42289beba7594bfa06e1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.552620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.637118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd4e8-11ca-4adf-a471-2333a6fb814c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.646624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.647477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.647859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.648220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.668388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77d6846b6df42289beba7594bfa06e1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:26.674391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.676219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12bb4-cbe3-4fbe-80ea-0bb7c6bb6e2c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.688460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77d6846b6df42289beba7594bfa06e1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:26.689286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77d6846b6df42289beba7594bfa06e1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:26.697197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 930/1881] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:26.709984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77d6846b6df42289beba7594bfa06e1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:26.723804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77d6846b6df42289beba7594bfa06e1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:26.736478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 952/1882] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:26 2022] DELETE /identity/v3/users/d77d6846b6df42289beba7594bfa06e1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:26.741490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.741868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.742315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.753605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:26.770616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.772121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.773586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:26.775798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.838254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:26.846749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 931/1883] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:26.860110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:26.862630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.865683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164f183-666d-4ec1-9fd6-f90dfb5d5ace admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.871681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.872363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.873409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.873875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.877107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:26.947951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daacf3-b1db-4d7d-ab4a-08c0e9bcf571 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:26.953896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:26.954674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:26.955067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:26.955448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:26.968794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.971350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.974415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:26.976268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:26.985815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 932/1884] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:42:26.989313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 953/1885] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:27.452756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:27.454957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:27.482637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02919d-e58a-4494-949a-32ac777948cb service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:27.489804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:27.491439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:27.491967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:27.492662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] PATH_INFO: `/v3/projects/bd2861392d3c409fa30edd4f2ebbc37d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:27.494362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2861392d3c409fa30edd4f2ebbc37d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:27.497178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:27.593855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2861392d3c409fa30edd4f2ebbc37d', 'request_id': 'req-2e00756d-ef96-45a7-8621-983f6b3bdeed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.608741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2861392d3c409fa30edd4f2ebbc37d', 'request_id': 'req-2e00756d-ef96-45a7-8621-983f6b3bdeed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.616999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2861392d3c409fa30edd4f2ebbc37d', 'request_id': 'req-2e00756d-ef96-45a7-8621-983f6b3bdeed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.630186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] The token cache is being invalidate because project bd2861392d3c409fa30edd4f2ebbc37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:27.630479 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.631166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.633013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 933/1886] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:27 2022] DELETE /identity/v3/projects/bd2861392d3c409fa30edd4f2ebbc37d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:27.636823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:27.647900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795e623-9432-4891-bc8d-9be7109ebd9f service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:27.718346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:27.719205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:27.719966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:27.719966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] PATH_INFO: `/v3/users/4a60a304ff894cb38eaa6fa55a2d464e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:27.720989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a60a304ff894cb38eaa6fa55a2d464e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:27.723308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:27.784336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a60a304ff894cb38eaa6fa55a2d464e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.796958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a60a304ff894cb38eaa6fa55a2d464e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.797678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a60a304ff894cb38eaa6fa55a2d464e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.806526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a60a304ff894cb38eaa6fa55a2d464e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.815618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc1666aa-9abb-4d50-86ea-20dd070e4357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a60a304ff894cb38eaa6fa55a2d464e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:27.821950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 954/1887] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:27 2022] DELETE /identity/v3/users/4a60a304ff894cb38eaa6fa55a2d464e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:28.095842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.097892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:28.112424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00756d-ef96-45a7-8621-983f6b3bdeed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.149238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:28.180121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.181078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.181364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.181636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] PATH_INFO: `/v3/projects/9fcae4f6fb9142d78ceb270a0476694f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.183094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcae4f6fb9142d78ceb270a0476694f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.185186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.239527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcae4f6fb9142d78ceb270a0476694f', 'request_id': 'req-5d0887e4-575a-4ead-b79d-67ee04a8f61b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:28.258671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcae4f6fb9142d78ceb270a0476694f', 'request_id': 'req-5d0887e4-575a-4ead-b79d-67ee04a8f61b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:28.265576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcae4f6fb9142d78ceb270a0476694f', 'request_id': 'req-5d0887e4-575a-4ead-b79d-67ee04a8f61b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:28.278603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] The token cache is being invalidate because project 9fcae4f6fb9142d78ceb270a0476694f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:28.278967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:28.279708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d0887e4-575a-4ead-b79d-67ee04a8f61b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:28.281415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 934/1888] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:28 2022] DELETE /identity/v3/projects/9fcae4f6fb9142d78ceb270a0476694f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:28.286927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.287425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.287678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.306345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:28.307610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:28.313201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:28.372016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 955/1889] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:28.379912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.380205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.380483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.402133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:28.403378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:28.412577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 935/1890] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:28.419115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.433467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44545700-4fa7-4a91-8af8-529bddd77582 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.441008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.441967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.442256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.442553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.443710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:28.444108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.446568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.462965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 956/1891] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:28.467524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.477867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abdd1f-1556-4a6c-b7ef-e825354064a7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.483712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.484415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.484769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.485103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.485865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.488121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.517851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 936/1892] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:28 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:28.527755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.538081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e775d-7df0-435e-8115-36faba57f5e9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.611981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.613256 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.613724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.614143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.615388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.618820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.645760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 957/1893] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:28 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:28.650791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.664858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd989c8-b84a-4384-bb6e-0645a57e53f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.671648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.672301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.672585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.672869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.673613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.675307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.687122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 937/1894] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:28.691144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.701604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e2eb9-1815-4048-9654-f90cd75fe152 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.710050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.710896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.711269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.711551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.712463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.715367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f329e2-e7b7-43dc-beb0-d184f8e2194b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.739859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:28.741154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 958/1895] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:28.776456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28f11e-391b-4b37-b821-b83e7edbeb7a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:28.789423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:28.790418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.790697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.791651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] PATH_INFO: `/v3/projects/254289a2466e494c954378dfaa16c972/users/5cd4915fa6ae4c44a0b04c0c664f2892/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.815248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] RBAC: Authorizing `identity:create_grant(project_id=254289a2466e494c954378dfaa16c972, user_id=5cd4915fa6ae4c44a0b04c0c664f2892, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:28.819442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:28.849362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 938/1896] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:28 2022] PUT /identity/v3/projects/254289a2466e494c954378dfaa16c972/users/5cd4915fa6ae4c44a0b04c0c664f2892/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:28.853262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:28.853771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:28.856733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:28.921121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] MFA Rules not processed for user `5cd4915fa6ae4c44a0b04c0c664f2892`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:28.923654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:29.089214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 959/1897] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:29.094078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:29.218757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:29.226176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8c5f-ba21-43fa-b058-d7e9006832da admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:29.240961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:29.241743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.242022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.242364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.246295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:29.248298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:29.318318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 939/1898] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:29 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:29.325841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:29.343981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c1f46-938e-417b-8092-273e4de9c117 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:29.354626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:29.501826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:29.502687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.502977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.503288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.503988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:29.514586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:29.564610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 960/1899] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:29 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:29.569876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:29.584146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d51f26-814a-4b04-81b0-7d1adcc655b1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:29.600365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:29.603585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.603980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.604674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.607846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:29.610015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:29.639389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 940/1900] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:29.644666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:29.663648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09d92-5f4d-45ce-bf9a-a4cab17758e0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:29.675371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:29.677549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.677875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.678226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.681199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:29.685737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e33f1-e926-40d6-b462-85511f01277f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:29.716625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 961/1901] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:29.718189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:29.733452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519fefd-173d-4756-9b2b-e96f59842337 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:29.763994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:29.765227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.765514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.765793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] PATH_INFO: `/v3/projects/ce2c7dbda59546f5a2f2d0d9513c78f4/users/2d5bf50338e044919afd3520d92e1940/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.799207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2c7dbda59546f5a2f2d0d9513c78f4, user_id=2d5bf50338e044919afd3520d92e1940, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:29.802175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:29.827854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 941/1902] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:29 2022] PUT /identity/v3/projects/ce2c7dbda59546f5a2f2d0d9513c78f4/users/2d5bf50338e044919afd3520d92e1940/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:29.839620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:29.840208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:29.841155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:29.885038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] MFA Rules not processed for user `2d5bf50338e044919afd3520d92e1940`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:29.886303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:29.969414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 962/1903] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:29.975042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.112615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dde7c-4a21-4e54-98e3-0f20a02e0f68 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.118648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.119665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.119957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.120241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.121004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.123916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.196809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 942/1904] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:29 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:30.198061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.220133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd6bfa-458e-457f-bd5b-52ebdec9a209 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.442964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.443766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.444061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.444391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.445199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.448337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.489186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 963/1905] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:30 2022] POST /identity/v3/users => generated 295 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:30.493074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.518706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64b85a-bbd4-4e9e-9ccb-a8a0816212eb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.544434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.545042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.545301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.545636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.546392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.548571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.574962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 943/1906] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:30.579316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.587203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:30.598642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08428b-263b-4a2b-9114-0d38ad87ed5c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.606217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.608745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.609878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.610380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] PATH_INFO: `/v3/projects/274741d6a0f3444e8f50fe6c86af7aef/users/e65be40b45234241a4f3e6c4ee2578e3/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.642868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=274741d6a0f3444e8f50fe6c86af7aef, user_id=e65be40b45234241a4f3e6c4ee2578e3, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.647044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf43a8-3ef2-434c-9315-cab8c11ffd3c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.686994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 964/1907] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:30 2022] PUT /identity/v3/projects/274741d6a0f3444e8f50fe6c86af7aef/users/e65be40b45234241a4f3e6c4ee2578e3/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:30.689155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.700105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcf397-f9f4-4696-b9e9-2e090a38a4a4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.827731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.828439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.828731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.829012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.829752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.832108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97237cb-0b17-4063-bdd5-e71dfc80b2e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.847660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952c2c-3664-4a8c-8954-5968f7888b09 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.848359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952c2c-3664-4a8c-8954-5968f7888b09 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.848792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952c2c-3664-4a8c-8954-5968f7888b09 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.849511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 944/1908] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:30.850714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 965/1909] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:42:30.854691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86952c2c-3664-4a8c-8954-5968f7888b09 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:30.880253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86952c2c-3664-4a8c-8954-5968f7888b09 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:30.890736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:30.891637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.891940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.892224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.892486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.892934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.893375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] PATH_INFO: `/v3/projects/274741d6a0f3444e8f50fe6c86af7aef/users/e65be40b45234241a4f3e6c4ee2578e3/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.902082 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=274741d6a0f3444e8f50fe6c86af7aef, user_id=e65be40b45234241a4f3e6c4ee2578e3, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:30.905174 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871ac80-345e-4eaf-8b98-c95a0d52c4dd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:30.928384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:30.929649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:30.933533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 966/1910] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:30 2022] PUT /identity/v3/projects/274741d6a0f3444e8f50fe6c86af7aef/users/e65be40b45234241a4f3e6c4ee2578e3/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:30.944295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:30.944869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:30.945355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:30.966038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:31.023275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] MFA Rules not processed for user `e65be40b45234241a4f3e6c4ee2578e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:31.025313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.071846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 945/1911] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:42:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:31.084797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:31.086500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.119180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4e32d-41ca-4f15-a08d-d2f24deac1f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:31.140719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:31.141957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.142436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.142913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.155371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:31.160058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:31.187405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 967/1912] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:31.194035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 946/1913] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:31.203142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:31.205244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.254032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:31.256284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.310129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b2c75-ea9f-41a2-8e4e-554734bba22b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:31.316630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:31.317311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.317597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.317874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.332079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:31.333815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:31.346377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 968/1914] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:31.383375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba415abc-6b0f-4682-99b1-041c73b9d954 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:31.392274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:31.393371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.393749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.394166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] PATH_INFO: `/v3/users/8588cad0074c4b958813977a4c5366cd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.401571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8588cad0074c4b958813977a4c5366cd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:31.403589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:31.415932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:31.417503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.439464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e38942-29e4-4bc1-a6e7-b070470451a7 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:31.445423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:31.446506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.446806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.447108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.497096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8588cad0074c4b958813977a4c5366cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:31.509229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8588cad0074c4b958813977a4c5366cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:31.516313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:31.520349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:31.521104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8588cad0074c4b958813977a4c5366cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:31.521806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8588cad0074c4b958813977a4c5366cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:31.521943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5937e1-04cf-4d81-b5f4-3ead6419ceda service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:31.522613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8588cad0074c4b958813977a4c5366cd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:31.535253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 969/1915] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:42:31.556655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 947/1916] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:31 2022] DELETE /identity/v3/users/8588cad0074c4b958813977a4c5366cd => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:31.560679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.563341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.563797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.584117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:31.585855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.739958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 970/1917] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:31.748696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:31.750613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:31.778016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:31.852658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6fb05-914d-4e96-90c0-c94816b48ca9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:31.858974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:31.859661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:31.860215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:31.860632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:31.872810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:31.874335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:31.886401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 948/1918] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:32.200870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:32.202683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:32.216204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187671a4-d7e2-48b1-a328-1c0d493bf99b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:32.221917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:32.222901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:32.223278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:32.223621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] PATH_INFO: `/v3/projects/40230e4314e449339e244c3e69b0edce` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:32.225043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40230e4314e449339e244c3e69b0edce)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:32.227202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:32.280499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40230e4314e449339e244c3e69b0edce', 'request_id': 'req-8dc03053-99a4-4949-ad07-4c652d880f38'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.286774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40230e4314e449339e244c3e69b0edce', 'request_id': 'req-8dc03053-99a4-4949-ad07-4c652d880f38'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.297306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40230e4314e449339e244c3e69b0edce', 'request_id': 'req-8dc03053-99a4-4949-ad07-4c652d880f38'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.308377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] The token cache is being invalidate because project 40230e4314e449339e244c3e69b0edce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:32.308723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.309419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc03053-99a4-4949-ad07-4c652d880f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.311034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 971/1919] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:32 2022] DELETE /identity/v3/projects/40230e4314e449339e244c3e69b0edce => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:32.315289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:32.325936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf01a7-7a80-4553-968a-d35bc21ed5ca service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:32.387243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:32.388139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:32.388418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:32.388685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] PATH_INFO: `/v3/users/75e066765196408c810545f392588ded` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:32.389896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e066765196408c810545f392588ded)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:32.391897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:32.441118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e066765196408c810545f392588ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.448727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e066765196408c810545f392588ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.458415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e066765196408c810545f392588ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.459095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e066765196408c810545f392588ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.459739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-391997dc-9e5e-4e77-8b77-77babcda1889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e066765196408c810545f392588ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:32.477220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 949/1920] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:32 2022] DELETE /identity/v3/users/75e066765196408c810545f392588ded => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:33.184253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:33.184833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:33.184951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:33.186438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 972/1921] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:42:33.215917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37389f-a151-44e0-b48c-f9c2dce8575c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:33.216478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37389f-a151-44e0-b48c-f9c2dce8575c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:33.216478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37389f-a151-44e0-b48c-f9c2dce8575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:33.254565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1f37389f-a151-44e0-b48c-f9c2dce8575c None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:33.256120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f37389f-a151-44e0-b48c-f9c2dce8575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:33.324703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:33.412681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 950/1922] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:42:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:33.961376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:33.964486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.058808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f634c0-e98b-45b0-a508-c21cce944b45 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.068325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.069234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.069520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.069857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] PATH_INFO: `/v3/projects/3b38f521c99a40e18c94515702f296d0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.071744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b38f521c99a40e18c94515702f296d0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.074193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.103616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:34.145106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b38f521c99a40e18c94515702f296d0', 'request_id': 'req-74cff59f-2d67-47d3-adad-0375302c7dd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.152312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b38f521c99a40e18c94515702f296d0', 'request_id': 'req-74cff59f-2d67-47d3-adad-0375302c7dd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.169032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b38f521c99a40e18c94515702f296d0', 'request_id': 'req-74cff59f-2d67-47d3-adad-0375302c7dd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.183757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] The token cache is being invalidate because project 3b38f521c99a40e18c94515702f296d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:34.184288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.185478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.187898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 973/1923] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:33 2022] DELETE /identity/v3/projects/3b38f521c99a40e18c94515702f296d0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:34.193632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d8daa-3922-4c91-945f-ca8d372cc075 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.194193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d8daa-3922-4c91-945f-ca8d372cc075 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.194468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d8daa-3922-4c91-945f-ca8d372cc075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.217358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1e1d8daa-3922-4c91-945f-ca8d372cc075 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:34.219772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e1d8daa-3922-4c91-945f-ca8d372cc075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.293046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.294493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.319793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 951/1924] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:34.329663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.329663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.329949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.348846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:34.350079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.360716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 952/1925] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:34.364452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.376302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aaa94d-2784-4e42-865a-a4a4e089eb66 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.382441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.383682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.384004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.384285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.385198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:34.385520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.387234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.393508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff59f-2d67-47d3-adad-0375302c7dd9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.401698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.402626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.402989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.403515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.403617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 953/1926] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:42:34.405247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.408017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.425866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.425866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.426152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9752fd0-5ad7-4bc7-8ade-fd793c5a7344 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.435858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.436208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 974/1927] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:34.436920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.437495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.437784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.439226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.441325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.475869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 954/1928] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:34.480135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.495845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0b0d6-58c5-45f5-a1cd-70ed75347ac9 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.591302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.592096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.592451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.592796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.593578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.595863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.629555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 975/1929] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:34.633643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.645262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73b033-2f33-43c9-a781-b623b1f9b302 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.650394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.651210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.651500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.651821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.652514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.654141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.668265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 955/1930] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:34.669726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.680093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a7eb-39b2-4aca-bc6f-48e3e9091b8f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.685392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.686091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.686377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.686742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.687807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.689723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.704291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 976/1931] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:34.705636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.718024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e2b78-158b-4a06-a29c-2570a2cf3e85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.723796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.724817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.725101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.725432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] PATH_INFO: `/v3/projects/c96fc25d7e1e49d398046ebb91085ad4/users/50838f6bb1554654b3628f90b041592a/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.738390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.740302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.750932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96fc25d7e1e49d398046ebb91085ad4, user_id=50838f6bb1554654b3628f90b041592a, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.754429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5923b6-2245-429a-a712-5132a9284ed7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.757015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b04e1-dadc-4c3b-a80a-7294972165e3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:34.763687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:34.765354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.765354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.765354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] PATH_INFO: `/v3/users/3807b84b72a14926aa71c5b562b8e1ca` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.767363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3807b84b72a14926aa71c5b562b8e1ca)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:34.770137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:34.776334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:34.780469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 956/1932] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:34 2022] PUT /identity/v3/projects/c96fc25d7e1e49d398046ebb91085ad4/users/50838f6bb1554654b3628f90b041592a/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:34.791817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.792146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.792585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:34.834844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] MFA Rules not processed for user `50838f6bb1554654b3628f90b041592a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:34.835941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:34.888442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3807b84b72a14926aa71c5b562b8e1ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.898557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:34.921220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3807b84b72a14926aa71c5b562b8e1ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.923620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3807b84b72a14926aa71c5b562b8e1ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.938330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3807b84b72a14926aa71c5b562b8e1ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.949085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b0ced220-7dae-4942-a892-fe4037850dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3807b84b72a14926aa71c5b562b8e1ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:34.955940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 957/1933] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:34.960772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:34.971861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 977/1934] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:34 2022] DELETE /identity/v3/users/3807b84b72a14926aa71c5b562b8e1ca => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:34.992188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:34.993638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:34.993931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.022652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:35.026254 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.150082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a46d54-6582-40cd-ba87-035cfad26429 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.181236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 978/1935] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:35.196540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.203272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.360351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.362959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.363194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.363567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.364353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.371367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe9b7-5cb2-4570-a565-1c60ceeb0fa2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.376290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.380719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.383620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.384102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.384548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.412275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.413958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.435827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 979/1936] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:35.436978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.438518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.460477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 958/1937] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:34 2022] POST /identity/v3/projects => generated 389 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:35.461847 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5647f-a645-4c32-8757-8d36e7e13978 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.463330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.466604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.501269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:35.574168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.576512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.576632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.576893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] PATH_INFO: `/v3/users/5cd4915fa6ae4c44a0b04c0c664f2892` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.579659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd4915fa6ae4c44a0b04c0c664f2892)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.584889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.637847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6038f-7a79-4bcf-ad26-a4df5fb958f7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.658084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.658797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.659138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.659404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.660121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.663680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.703345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd4915fa6ae4c44a0b04c0c664f2892'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.715621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 959/1938] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:35 2022] POST /identity/v3/users => generated 301 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:35.716854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.723799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.737465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd4915fa6ae4c44a0b04c0c664f2892'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.738454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd4915fa6ae4c44a0b04c0c664f2892'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.743999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:35.748624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f129a-8cee-426c-9ec5-3dc23c1cfede admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.755217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.755811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd4915fa6ae4c44a0b04c0c664f2892'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.765192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.766398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.766943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd4915fa6ae4c44a0b04c0c664f2892'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.767100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] PATH_INFO: `/v3/users/cd66084e257b4f65b74407899b8c9614` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.768855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd66084e257b4f65b74407899b8c9614)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.772935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.778106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 980/1939] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:35 2022] DELETE /identity/v3/users/5cd4915fa6ae4c44a0b04c0c664f2892 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:35.779448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.780728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.848541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd66084e257b4f65b74407899b8c9614'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.856469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd66084e257b4f65b74407899b8c9614'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.863580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0209e-6cb8-4389-bf7f-7ede58862f91 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.865896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd66084e257b4f65b74407899b8c9614'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.866734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd66084e257b4f65b74407899b8c9614'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.867435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd66084e257b4f65b74407899b8c9614'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:35.869486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.870321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.871162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.871452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.872191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:35.873977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cad5d-a6cf-4a10-af87-2b91af0a25ef admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:35.887857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 981/1940] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:35.887857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbdbc-91e5-4644-bfa7-c15680c64f78 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.887857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbdbc-91e5-4644-bfa7-c15680c64f78 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.887857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cbdbc-91e5-4644-bfa7-c15680c64f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.888245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 960/1941] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:35 2022] DELETE /identity/v3/users/cd66084e257b4f65b74407899b8c9614 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:35.890513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:35.891973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.908222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-108cbdbc-91e5-4644-bfa7-c15680c64f78 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:35.909460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-108cbdbc-91e5-4644-bfa7-c15680c64f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:35.959725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:35.974545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37e8f4-378c-4dd7-9ba2-42b87c64cbd4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:35.979466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:35.980391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.980682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.980968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] PATH_INFO: `/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb/users/6a1cb152819142dcae7b0d906af25a07/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:35.985830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 982/1942] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:35.987255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:35.988015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:35.988015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.005535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:36.006752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.007720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d3e5b980014ca3a0f163b97d382bfb, user_id=6a1cb152819142dcae7b0d906af25a07, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.010607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.014582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 983/1943] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:36.018329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.019443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.028819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 961/1944] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:35 2022] PUT /identity/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb/users/6a1cb152819142dcae7b0d906af25a07/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:36.029490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.031298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.034026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e42dc-7aa2-43fb-8702-87fbdd9ee53a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.092418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e569f-1b0f-47fe-b638-7a3e6be2c35f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.096467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.097220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.097507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.097786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.099920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.100809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.101150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.101363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] PATH_INFO: `/v3/projects/8ffbc6eab1bc433ab46a95ea589c7dfe` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.102594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffbc6eab1bc433ab46a95ea589c7dfe)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.104842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.108870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.110380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.123768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 962/1945] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:36.124749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.127641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.134799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:36.140522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dd12-9f71-4a78-86eb-7671112ff9bb service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.148622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.151312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.152297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.152953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.169123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffbc6eab1bc433ab46a95ea589c7dfe', 'request_id': 'req-4bf53b3c-7360-420d-ae71-bd497a38478d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:36.172289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.173991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.177515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffbc6eab1bc433ab46a95ea589c7dfe', 'request_id': 'req-4bf53b3c-7360-420d-ae71-bd497a38478d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:36.188574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 963/1946] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:36.189443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.190968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffbc6eab1bc433ab46a95ea589c7dfe', 'request_id': 'req-4bf53b3c-7360-420d-ae71-bd497a38478d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:36.208497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] The token cache is being invalidate because project 8ffbc6eab1bc433ab46a95ea589c7dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:36.209123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:36.209234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e12ab5-1958-4ea6-8181-2f3295272eaa service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.209413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4bf53b3c-7360-420d-ae71-bd497a38478d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:36.211347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 984/1947] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:36 2022] DELETE /identity/v3/projects/8ffbc6eab1bc433ab46a95ea589c7dfe => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:36.216719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341829-a07e-403b-91b9-df9377712df3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.217253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341829-a07e-403b-91b9-df9377712df3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.217548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341829-a07e-403b-91b9-df9377712df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.240422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-81341829-a07e-403b-91b9-df9377712df3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:36.241497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-81341829-a07e-403b-91b9-df9377712df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.319147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.321804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.322409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.322898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.324789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.327740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475f01-a093-48f9-8acf-634da2085071 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.345052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 964/1948] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:36.345266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 985/1949] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:36.345352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.345352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.345648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.346187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81341829-a07e-403b-91b9-df9377712df3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.347648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-81341829-a07e-403b-91b9-df9377712df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.365067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81341829-a07e-403b-91b9-df9377712df3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.370653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.371612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.371950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.372281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] PATH_INFO: `/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb/users/6a1cb152819142dcae7b0d906af25a07/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.372596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:36.373986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.374918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d3e5b980014ca3a0f163b97d382bfb, user_id=6a1cb152819142dcae7b0d906af25a07, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.378651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2c88-a3b2-4e7f-857b-f9d2501f3ecc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.381686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:36.389862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 965/1950] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:36.394257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.399901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 986/1951] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:36 2022] PUT /identity/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb/users/6a1cb152819142dcae7b0d906af25a07/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:36.407767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e97750-3f32-4ac3-9075-8d13b938c763 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.412236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.412589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.412968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.454272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] MFA Rules not processed for user `6a1cb152819142dcae7b0d906af25a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:36.454687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.496179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.497311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.497626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.498005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.499276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:36.499695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.501801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.519404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 966/1952] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:36.523697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.537010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706074c-a84b-418e-80a4-2f1138fac622 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.545312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.546727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.549253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.549253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.549639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.555560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.580895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 987/1953] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:36.595676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.597269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:36.612118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 967/1954] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:36.614401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.684801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:36.688685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580f38-d8ab-4cf3-b7eb-fbfe0dfe6720 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.700697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377ca-aa5e-4618-87ae-add107337f43 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.706420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.707420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.707709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.707991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.709570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.711941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.739129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 968/1955] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:36.746979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.761924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299956-6bb7-4dd7-83e6-facc43651160 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.771766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.772813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.772951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.773448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.774258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.774603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.774917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.775303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.776745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.780608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.795739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 969/1956] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:36.800389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.815178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcee953-6d2b-4167-8fab-81ab6623c731 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.823263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.824723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.825337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.825551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.826394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.828720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.838971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:36.847314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 970/1957] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:36.852352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:36.861733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.864299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ca50f-e9ee-4502-ab87-69c6986fdaac service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.867992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b0ec7-a349-4a0d-b7b1-1bd2eb6cf02a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:36.875804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:36.876849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.877384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.877795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 988/1958] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:36.879342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] PATH_INFO: `/v3/projects/2334b95fd02b47b1b354dc8c7fed8ec4/users/1b8573a17f1b449a8040efb4e7918381/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.904086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2334b95fd02b47b1b354dc8c7fed8ec4, user_id=1b8573a17f1b449a8040efb4e7918381, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:36.904086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:36.920630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 971/1959] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:36 2022] PUT /identity/v3/projects/2334b95fd02b47b1b354dc8c7fed8ec4/users/1b8573a17f1b449a8040efb4e7918381/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:36.931743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:36.932031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:36.932312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:36.980330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] MFA Rules not processed for user `1b8573a17f1b449a8040efb4e7918381`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:36.983680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:37.081148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:37.109181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 989/1960] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:37.124361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:37.205396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5fd63-f515-4f0f-a749-e955ca27ceb4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:37.213260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:37.214259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:37.214637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:37.215545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] PATH_INFO: `/v3/users/50838f6bb1554654b3628f90b041592a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:37.216943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50838f6bb1554654b3628f90b041592a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:37.219371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:37.222911 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc84bf-9135-4a58-92dc-7d6a89eca654 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:37.234925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001689-608d-40fc-9f37-09204742029f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:37.236568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001689-608d-40fc-9f37-09204742029f service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:37.237038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001689-608d-40fc-9f37-09204742029f service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:37.237969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001689-608d-40fc-9f37-09204742029f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:37.254535 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001689-608d-40fc-9f37-09204742029f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:37.256698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001689-608d-40fc-9f37-09204742029f service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:37.269320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 990/1961] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:37.288561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50838f6bb1554654b3628f90b041592a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.296902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50838f6bb1554654b3628f90b041592a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.310321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50838f6bb1554654b3628f90b041592a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.311887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50838f6bb1554654b3628f90b041592a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.312598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50838f6bb1554654b3628f90b041592a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.353469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 972/1962] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:37 2022] DELETE /identity/v3/users/50838f6bb1554654b3628f90b041592a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:37.358369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:37.358662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:37.358942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:37.388913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:37.390521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:37.480924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 991/1963] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:37.494215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:37.496428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:37.523240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:37.617546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c560459-d53b-4477-bd3c-fecd1653ce26 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:37.622114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:37.622944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:37.624590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:37.624590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:37.642262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:37.644882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:37.660706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 973/1964] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:37.755994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:37.758313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:37.786220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e65a5-99a6-42b5-b2ac-7b1f229b5101 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:37.797386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:37.798454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:37.799530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:37.799958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] PATH_INFO: `/v3/projects/33dfd41a1d2747d0b41cf80a107c9e35` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:37.801683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33dfd41a1d2747d0b41cf80a107c9e35)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:37.806891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:37.842765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:37.929821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33dfd41a1d2747d0b41cf80a107c9e35', 'request_id': 'req-40e372d7-8d48-48e7-acc7-c898703e01e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.938156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33dfd41a1d2747d0b41cf80a107c9e35', 'request_id': 'req-40e372d7-8d48-48e7-acc7-c898703e01e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.949319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33dfd41a1d2747d0b41cf80a107c9e35', 'request_id': 'req-40e372d7-8d48-48e7-acc7-c898703e01e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.968102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] The token cache is being invalidate because project 33dfd41a1d2747d0b41cf80a107c9e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:37.968589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.969817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:37.971900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 992/1965] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:37 2022] DELETE /identity/v3/projects/33dfd41a1d2747d0b41cf80a107c9e35 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:37.976188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:37.989883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8b141-aaec-4fd9-8a28-85d271f55d97 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.067768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:38.069269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:38.119440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.122218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.123374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.123678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] PATH_INFO: `/v3/users/74a0aa6368874de0a8790001fbe8dd1c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.125187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a0aa6368874de0a8790001fbe8dd1c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:38.128109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:38.182671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e372d7-8d48-48e7-acc7-c898703e01e4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.188432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.189315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.189600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.189927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] PATH_INFO: `/v3/projects/254289a2466e494c954378dfaa16c972` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.191239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=254289a2466e494c954378dfaa16c972)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:38.193607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:38.222565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a0aa6368874de0a8790001fbe8dd1c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.233004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a0aa6368874de0a8790001fbe8dd1c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.246119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a0aa6368874de0a8790001fbe8dd1c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.246825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a0aa6368874de0a8790001fbe8dd1c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.247667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a0aa6368874de0a8790001fbe8dd1c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.269198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254289a2466e494c954378dfaa16c972', 'request_id': 'req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.277331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 974/1966] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:37 2022] DELETE /identity/v3/users/74a0aa6368874de0a8790001fbe8dd1c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:38.286583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254289a2466e494c954378dfaa16c972', 'request_id': 'req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.297073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254289a2466e494c954378dfaa16c972', 'request_id': 'req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.314276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] The token cache is being invalidate because project 254289a2466e494c954378dfaa16c972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:38.314690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.315589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.317889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 993/1967] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:38 2022] DELETE /identity/v3/projects/254289a2466e494c954378dfaa16c972 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:38.322143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:38.324237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:38.329412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:38.330993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:38.396996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:38.435166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:38.447200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342c69d-bc71-4593-b4c1-99e50dc4f4b7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.491294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6da5a-c26f-412f-986b-d19fea9e119e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.498164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.499107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.499510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.499787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] PATH_INFO: `/v3/users/2d5bf50338e044919afd3520d92e1940` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.501029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5bf50338e044919afd3520d92e1940)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:38.512802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:38.652222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.656134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.657230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.658722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] PATH_INFO: `/v3/projects/c96fc25d7e1e49d398046ebb91085ad4` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.659390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5bf50338e044919afd3520d92e1940'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.662720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96fc25d7e1e49d398046ebb91085ad4)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:38.666293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:38.678841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5bf50338e044919afd3520d92e1940'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.702429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5bf50338e044919afd3520d92e1940'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.706893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5bf50338e044919afd3520d92e1940'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.708022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5bf50338e044919afd3520d92e1940'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.744100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 975/1968] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:38 2022] DELETE /identity/v3/users/2d5bf50338e044919afd3520d92e1940 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:38.745284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:38.749084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:38.774962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96fc25d7e1e49d398046ebb91085ad4', 'request_id': 'req-6cfb6974-d969-4c88-b330-501f895186de'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.787842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96fc25d7e1e49d398046ebb91085ad4', 'request_id': 'req-6cfb6974-d969-4c88-b330-501f895186de'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.813462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96fc25d7e1e49d398046ebb91085ad4', 'request_id': 'req-6cfb6974-d969-4c88-b330-501f895186de'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.844762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] The token cache is being invalidate because project c96fc25d7e1e49d398046ebb91085ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:38.845311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.845963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:38.852297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 994/1969] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:38 2022] DELETE /identity/v3/projects/c96fc25d7e1e49d398046ebb91085ad4 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:38.853160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:38.856452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:38.901355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d2071-1b3f-498a-b9a4-ceee3de3b484 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.985101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.986164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.986520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.986873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] PATH_INFO: `/v3/projects/f4275e6efe4949bca6073f308ad4c061` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.988229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4275e6efe4949bca6073f308ad4c061)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:38.989336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6974-d969-4c88-b330-501f895186de admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:38.990523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:38.994486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:38.996929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:38.997851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:38.998135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:38.998468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] PATH_INFO: `/v3/users/6a1cb152819142dcae7b0d906af25a07` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:38.999765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1cb152819142dcae7b0d906af25a07)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.001814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.015851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:39.068220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4275e6efe4949bca6073f308ad4c061', 'request_id': 'req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.081771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4275e6efe4949bca6073f308ad4c061', 'request_id': 'req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.082258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1cb152819142dcae7b0d906af25a07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.090655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4275e6efe4949bca6073f308ad4c061', 'request_id': 'req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.102898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1cb152819142dcae7b0d906af25a07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.104297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1cb152819142dcae7b0d906af25a07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.110994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] The token cache is being invalidate because project f4275e6efe4949bca6073f308ad4c061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:39.111408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.112147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dc6e6aa3-c341-4098-b3e8-0468c6cc0a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.114077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 976/1970] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:38 2022] DELETE /identity/v3/projects/f4275e6efe4949bca6073f308ad4c061 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:39.118575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1cb152819142dcae7b0d906af25a07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.121233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.121526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.121861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.134569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b6d1cbbd-d809-4b77-85c4-454cfeade4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1cb152819142dcae7b0d906af25a07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.146626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 995/1971] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:38 2022] DELETE /identity/v3/users/6a1cb152819142dcae7b0d906af25a07 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:39.151414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:39.152787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.241885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 977/1972] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:39.243602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.243602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.244101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.267865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:39.269104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.277974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 996/1973] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:39.282392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.293109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f753884-d675-478a-a502-bbc056d151d9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.298010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.298706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.298990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.299327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.300057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:39.300371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.302533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.315511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 978/1974] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:39.320080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.330309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce534997-9d2e-478c-9f87-4fa52f36ac8d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.336555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.337529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.337785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.338114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.338832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.340932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.382694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 997/1975] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:39 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:39.384007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.396743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6bf6-df85-44c8-bf1e-21e18463c179 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.441581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.443393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.530197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.532834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.533432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.533893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.534809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.537021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.578866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 979/1976] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:39 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:39.582085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.598090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2acc4-085a-4012-94c6-9e0255d0a1bc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.605699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438568b9-ba34-4f39-ac48-788f3110f877 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.611515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.611515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.611515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.611933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.614295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.615308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.615596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.615866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] PATH_INFO: `/v3/projects/ce2c7dbda59546f5a2f2d0d9513c78f4` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.616168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.617088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2c7dbda59546f5a2f2d0d9513c78f4)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.618535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.619837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.621734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:39.641334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 980/1977] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:39.644927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.668576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff159d1-712d-4508-83da-eccbca2c04d0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.675701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.676393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.676677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.677007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.677749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.679728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.702119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2c7dbda59546f5a2f2d0d9513c78f4', 'request_id': 'req-e0816259-6d80-4c33-b5a0-78c577719255'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.705059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 981/1978] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:39.706442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.710357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2c7dbda59546f5a2f2d0d9513c78f4', 'request_id': 'req-e0816259-6d80-4c33-b5a0-78c577719255'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.720535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852216-c9ad-43e2-9695-de2259aea4e4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.722514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2c7dbda59546f5a2f2d0d9513c78f4', 'request_id': 'req-e0816259-6d80-4c33-b5a0-78c577719255'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.724021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:39.738101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] The token cache is being invalidate because project ce2c7dbda59546f5a2f2d0d9513c78f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:39.738440 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.739325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:39.741596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 998/1979] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:39 2022] DELETE /identity/v3/projects/ce2c7dbda59546f5a2f2d0d9513c78f4 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:39.746859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.748877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.792090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.793041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.793325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.793603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] PATH_INFO: `/v3/projects/cff9b2b9836b4af09eb6649d24b51d90/users/27816a06229541f88bdfca45f2c4c232/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.812907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff9b2b9836b4af09eb6649d24b51d90, user_id=27816a06229541f88bdfca45f2c4c232, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.815640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274196f9-0031-40bd-bcf0-472bbcf94190 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.832531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 982/1980] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:39 2022] PUT /identity/v3/projects/cff9b2b9836b4af09eb6649d24b51d90/users/27816a06229541f88bdfca45f2c4c232/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:39.837688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816259-6d80-4c33-b5a0-78c577719255 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:39.844504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.844667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.844892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.874911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] MFA Rules not processed for user `27816a06229541f88bdfca45f2c4c232`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:39.876235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.931636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:39.932615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:39.932899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:39.933236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] PATH_INFO: `/v3/users/e65be40b45234241a4f3e6c4ee2578e3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:39.937200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65be40b45234241a4f3e6c4ee2578e3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:39.939655 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:39.985362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 983/1981] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:39.986242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:39.987625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:39.999387 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65be40b45234241a4f3e6c4ee2578e3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.008096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4591d8b-414b-4817-8c34-23621cc5c462 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.016460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122209e4-d706-404a-a409-621285aa751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.017282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65be40b45234241a4f3e6c4ee2578e3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.018063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65be40b45234241a4f3e6c4ee2578e3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.018256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122209e4-d706-404a-a409-621285aa751d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.018511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122209e4-d706-404a-a409-621285aa751d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.018744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122209e4-d706-404a-a409-621285aa751d admin admin] PATH_INFO: `/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.020238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122209e4-d706-404a-a409-621285aa751d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d3e5b980014ca3a0f163b97d382bfb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.022531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122209e4-d706-404a-a409-621285aa751d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.027849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65be40b45234241a4f3e6c4ee2578e3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.036723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65be40b45234241a4f3e6c4ee2578e3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.043735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 999/1982] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:39 2022] DELETE /identity/v3/users/e65be40b45234241a4f3e6c4ee2578e3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:40.047626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.048853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.077127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:40.096429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d3e5b980014ca3a0f163b97d382bfb', 'request_id': 'req-122209e4-d706-404a-a409-621285aa751d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.102522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:40.119196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d3e5b980014ca3a0f163b97d382bfb', 'request_id': 'req-122209e4-d706-404a-a409-621285aa751d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.129100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d3e5b980014ca3a0f163b97d382bfb', 'request_id': 'req-122209e4-d706-404a-a409-621285aa751d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.143514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] The token cache is being invalidate because project c3d3e5b980014ca3a0f163b97d382bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:40.143978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.144789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-122209e4-d706-404a-a409-621285aa751d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.146494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 984/1983] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:39 2022] DELETE /identity/v3/projects/c3d3e5b980014ca3a0f163b97d382bfb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:40.152671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910bab7-1bdc-483d-8023-006354a66c62 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.152981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e50a4-8f3e-4052-809e-6d2e587681ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.153279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e50a4-8f3e-4052-809e-6d2e587681ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.153567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e50a4-8f3e-4052-809e-6d2e587681ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.173700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ea0e50a4-8f3e-4052-809e-6d2e587681ee None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.175118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ea0e50a4-8f3e-4052-809e-6d2e587681ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.216583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.217362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.217725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.218264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.218974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.221239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.248106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 985/1984] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:40.255804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.256178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.256519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1000/1985] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:40.256662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.260855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.266946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.277443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.278791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.348229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 986/1986] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:40.350612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb94e-f187-42ef-bf24-924ec79d79f9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.353154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.356225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.357306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.357576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.357876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.358828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.362136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.364071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff954e5-9751-46ae-a7e7-a49f68c46101 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.372156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.373347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.373764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.374209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.375381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:40.375865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.377861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.395532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 987/1987] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:40.395532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.395827 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.399529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1001/1988] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:40.399529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.411669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2e5e0-8f47-4e4c-8d72-7c3955997c32 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.414996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c96c2-2934-40b5-b5aa-32cdfff560d9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.416651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.417533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.417821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.418100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] PATH_INFO: `/v3/projects/274741d6a0f3444e8f50fe6c86af7aef` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.419305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=274741d6a0f3444e8f50fe6c86af7aef)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.421217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.421428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.421897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.422185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.422466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.423197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.425430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.461806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1002/1989] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:40.462933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.478263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274741d6a0f3444e8f50fe6c86af7aef', 'request_id': 'req-21699516-8849-4dc8-9551-059696c7ad2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.482559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:40.489500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274741d6a0f3444e8f50fe6c86af7aef', 'request_id': 'req-21699516-8849-4dc8-9551-059696c7ad2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.497742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274741d6a0f3444e8f50fe6c86af7aef', 'request_id': 'req-21699516-8849-4dc8-9551-059696c7ad2b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.498990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:40.513395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] The token cache is being invalidate because project 274741d6a0f3444e8f50fe6c86af7aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:40.514047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.515144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:40.517194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 988/1990] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:40 2022] DELETE /identity/v3/projects/274741d6a0f3444e8f50fe6c86af7aef => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:40.518135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.519889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.548795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b80587-e915-4526-9bd8-df3cb5c29f7d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.601586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21699516-8849-4dc8-9551-059696c7ad2b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.607586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.608386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.608632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.608824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.619421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.619421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d06ff3-3cbc-4e0a-8f99-c3c6660564ec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.626146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.626838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.627161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.627474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.628395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.630051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.637071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 989/1991] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:40.637071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.637071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.637071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.641299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1003/1992] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:40.642123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.652483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797f8d-7f3f-4124-8128-fe18007aa2a6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.654971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.656216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.657299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.658028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.658313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.658659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.659355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.660973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cb9be-8878-45b4-982d-1718420119bb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.665056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 990/1993] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:40.666041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.667518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.672162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1004/1994] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:40.676512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.676801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.677074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.679581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11859e3-fe5c-426f-b599-93487616f667 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.684486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.685161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.685564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.685848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.686583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.688279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.694994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.696553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.701648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 991/1995] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:40.701648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.706653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1005/1996] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:40.707419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.710009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66f7f-da83-4c70-9522-82eff58bf5f7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.715769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.716440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.716722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.716951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e710cc5-9561-4aac-9204-56b75e9fa6db None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.717085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.717713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.719362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.721512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.722400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.722694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.722999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] PATH_INFO: `/v3/projects/983f87973fb94a138dbd3ee463cb2225/users/7d3818b0b6e04850b9673f6e0defd5e8/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.730568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 992/1997] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:40.731965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.742549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aed0-3e4c-4fd6-96c8-2139b0a16114 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.742974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983f87973fb94a138dbd3ee463cb2225, user_id=7d3818b0b6e04850b9673f6e0defd5e8, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.746184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.747896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.749542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.749848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.750166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.750878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:40.751262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.752890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.763762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1006/1998] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:40 2022] PUT /identity/v3/projects/983f87973fb94a138dbd3ee463cb2225/users/7d3818b0b6e04850b9673f6e0defd5e8/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:40.764619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.768557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 993/1999] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:40.773039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:40.783379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfda57-4e41-4fd9-9d52-9bfd8f21d7f1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.829555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666caf1e-a367-4e23-9225-e39fa7149174 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:40.834663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.835615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.836126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.836285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] PATH_INFO: `/v3/projects/872cdff97d9747e8a907acc168204163/users/136b491666ea45ce8320bf3bee2147fd/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.845024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:40.845699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.845986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.846280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.846979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.848971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b159a3-97bc-4e94-9767-2c2b3541539c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.851974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872cdff97d9747e8a907acc168204163, user_id=136b491666ea45ce8320bf3bee2147fd, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:40.854945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4dc53-1cda-49f7-813f-ed9552c70d47 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:40.865085 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:40.870122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1007/2000] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:40 2022] PUT /identity/v3/projects/872cdff97d9747e8a907acc168204163/users/136b491666ea45ce8320bf3bee2147fd/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:40.872404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.873088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.873392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.877819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 994/2001] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:40.881421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:40.881785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:40.882133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:40.909211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] MFA Rules not processed for user `7d3818b0b6e04850b9673f6e0defd5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.910526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.918673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] MFA Rules not processed for user `136b491666ea45ce8320bf3bee2147fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:40.919954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:40.936861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:41.000993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1008/2002] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:41.000993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.001295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 995/2003] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:41.004446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.069456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f7881-f972-4e07-b5db-f7f46eac86fd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.075207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.075897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.076178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.076454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.077147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.078835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ae31a-165f-4098-ab46-bd1e483c33f6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.079111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.084692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.085379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.086257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.086257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.087235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.089366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.103340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1009/2004] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:40 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:41.104706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.107797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.122807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 996/2005] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:41.123849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.191182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eba5-edce-4c1e-9669-f1d9125b5682 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.198482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99838c94-1250-4f1b-a792-7d4a6b2e2f33 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.204433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.205844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.206131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.206409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.207179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.209316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.225586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 997/2006] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:41.225586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.241549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:41.246369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7a845-e5e4-40c5-ba55-73381ea91f5b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.251338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.252144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.252431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.252766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.253449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.255440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.265053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.266667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.267372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.267969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.281284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 998/2007] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/users => generated 300 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:41.282372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.294213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055b471-565c-4ad5-8a8d-ca96c70e582b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.300137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.300820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.301105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.301384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.302788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.304939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.322557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 999/2008] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:41.323485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.335455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e45f3-879f-431e-848c-aeef90f9f02c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.343443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.344286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.344579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.344882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.345684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.347323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.350816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:41.351991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.353744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.365543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1000/2009] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:41.366521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.367415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1010/2010] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:41.378309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b68fca-6696-4dbd-b649-727b79e56c7f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.380889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.384560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.385517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.385801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.386077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] PATH_INFO: `/v3/projects/68f1b01f384b4970bfeca47ff8629136/users/0da1a254911942dab58691f79b3151a1/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.406553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f1b01f384b4970bfeca47ff8629136, user_id=0da1a254911942dab58691f79b3151a1, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.409242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13bf41-7cfc-4337-80a3-6819f84c14c7 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.410021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.418274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.419298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.419591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.419873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] PATH_INFO: `/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e/users/da324f3736fb4f80ae45e0b1f1b3dfdc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.435222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1001/2011] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:41 2022] PUT /identity/v3/projects/68f1b01f384b4970bfeca47ff8629136/users/0da1a254911942dab58691f79b3151a1/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:41.435222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.445624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8ed9945d9b48c9826222a3a29ea14e, user_id=da324f3736fb4f80ae45e0b1f1b3dfdc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.449089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c543bc-4c18-4367-b26a-92fb8ae5da29 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.464306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1011/2012] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:41 2022] PUT /identity/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e/users/da324f3736fb4f80ae45e0b1f1b3dfdc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:41.468433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.468775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.469211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.498247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] MFA Rules not processed for user `0da1a254911942dab58691f79b3151a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:41.502629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.519690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7845f21-4959-4d9d-b738-cebe098a7542 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.582022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1012/2013] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:41.582711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.594028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.595112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.595919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.595919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] PATH_INFO: `/v3/users/136b491666ea45ce8320bf3bee2147fd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.596911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=136b491666ea45ce8320bf3bee2147fd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.598925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.656670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:41.661315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291550e3-fb62-4f1f-b695-2b99c9708801 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.667876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.668589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.668764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136b491666ea45ce8320bf3bee2147fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:41.669223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.669223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.669856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.671657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.675784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136b491666ea45ce8320bf3bee2147fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:41.686199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136b491666ea45ce8320bf3bee2147fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:41.687257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136b491666ea45ce8320bf3bee2147fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:41.688034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136b491666ea45ce8320bf3bee2147fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:41.692937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1013/2014] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:41.692937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.692937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.706140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9348e1e-8c32-40bc-a4fe-da0945c77e27 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.708882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:41.711810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.712447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.712739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.713029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.713737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.715019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1002/2015] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:41 2022] DELETE /identity/v3/users/136b491666ea45ce8320bf3bee2147fd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:41.716491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.717575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34a4c-9c79-4213-bc26-1a01b90cf125 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.719246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.753165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1014/2016] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:41.754571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.754876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.755188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.784593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:41.785931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.799948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761bf23-1318-41ef-8527-fcba9ddd4987 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.854361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1015/2017] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:41.855444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.858874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.859877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.860163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.860661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] PATH_INFO: `/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e/users/da324f3736fb4f80ae45e0b1f1b3dfdc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.864079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8ed9945d9b48c9826222a3a29ea14e, user_id=da324f3736fb4f80ae45e0b1f1b3dfdc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.865662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640859b-31d9-415a-ba74-a6e04be486e9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.866880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.870842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.871554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.871836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.872137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.872830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:41.874732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f961a41-f486-4b15-8320-dc7ca2256ef0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:41.878522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1003/2018] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:41 2022] PUT /identity/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e/users/da324f3736fb4f80ae45e0b1f1b3dfdc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:41.879201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:41.880948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.903115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1016/2019] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:41.904290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.904580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.904858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.933038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] MFA Rules not processed for user `da324f3736fb4f80ae45e0b1f1b3dfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:41.934244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:41.943679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867cd1-90a6-43e4-89d1-20f2577daffe admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:41.947516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:41.948174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:41.948457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:41.948736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:41.999946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:42.005001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1017/2020] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:42.005829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.016261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.016449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b82891-df95-48cb-ab24-f159abf8a79b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.017803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.020983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.021659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.021944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.022219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.022982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.024735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.028360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1004/2021] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:42.029247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.030651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:42.037813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1018/2022] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:42.038898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.050126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cc44a-a6ab-4cc5-9fd5-01b1db96175e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.055521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.056555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.056840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.057166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] PATH_INFO: `/v3/projects/4e80cd6012e54426a18e8ce8b731eba4/users/69f054c5dea44385bba6199c462f3748/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.077286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e80cd6012e54426a18e8ce8b731eba4, user_id=69f054c5dea44385bba6199c462f3748, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.080980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.100396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1019/2023] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:42 2022] PUT /identity/v3/projects/4e80cd6012e54426a18e8ce8b731eba4/users/69f054c5dea44385bba6199c462f3748/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:42.100608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cddc7-d31b-4c07-9fc9-a0f8b9b1328f service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.102010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.107425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:42.157880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.159894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.160889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.161461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.175257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d0c85-08a6-47c2-8b00-8cb67abdc78f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.182119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.183348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.183635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.183913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.184660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.186467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.204548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1020/2024] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:42.206806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.220353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec690a6-a519-4852-b5b4-d9199742ee7c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.226747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.228286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.228580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.229143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] PATH_INFO: `/v3/projects/4e80cd6012e54426a18e8ce8b731eba4/users/69f054c5dea44385bba6199c462f3748/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.232280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e80cd6012e54426a18e8ce8b731eba4, user_id=69f054c5dea44385bba6199c462f3748, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.236855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c13aa4-ec42-4b70-ac2b-ffcafaf2ad32 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.259162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1021/2025] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:42 2022] PUT /identity/v3/projects/4e80cd6012e54426a18e8ce8b731eba4/users/69f054c5dea44385bba6199c462f3748/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:42.259541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.259541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.262801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.263504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1005/2026] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:42.264852 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.265147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.314009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] MFA Rules not processed for user `69f054c5dea44385bba6199c462f3748`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:42.315423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:42.382686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.387199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:42.424808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773c412-90ef-4c29-8933-550eeaf81aea service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.523628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1022/2027] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:42.538466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:42.559687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.560638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.560914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.561220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] PATH_INFO: `/v3/projects/872cdff97d9747e8a907acc168204163` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.563935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=872cdff97d9747e8a907acc168204163)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.564755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.566649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.630042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526a7e5-ad23-427c-99a2-75d28b5544e3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:42.634967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:42.636188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.636529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.637025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.674189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:42.675667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9dc9-a8cf-422f-9455-aa2f31a20bc4 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:42.680302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872cdff97d9747e8a907acc168204163', 'request_id': 'req-25595369-fd9b-40e7-8061-b06eda62c096'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:42.708377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872cdff97d9747e8a907acc168204163', 'request_id': 'req-25595369-fd9b-40e7-8061-b06eda62c096'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:42.719834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1023/2028] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:42.730208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872cdff97d9747e8a907acc168204163', 'request_id': 'req-25595369-fd9b-40e7-8061-b06eda62c096'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:42.749929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] The token cache is being invalidate because project 872cdff97d9747e8a907acc168204163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:42.750274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:42.751183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:42.753377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1006/2029] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:42 2022] DELETE /identity/v3/projects/872cdff97d9747e8a907acc168204163 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:42.759760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722f8a1-16f5-402c-8187-ce47e12e531d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:42.760084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722f8a1-16f5-402c-8187-ce47e12e531d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:42.760378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722f8a1-16f5-402c-8187-ce47e12e531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:42.843574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a722f8a1-16f5-402c-8187-ce47e12e531d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:42.844580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a722f8a1-16f5-402c-8187-ce47e12e531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:42.890070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:42.946343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:42.948620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.010662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1024/2030] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:43.018441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.018807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.019186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.060871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:43.062185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.088242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1025/2031] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:43.093584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.110379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25595369-fd9b-40e7-8061-b06eda62c096 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.120460 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.122855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.124202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2899-468a-40cb-be86-0121d430d245 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.124517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.124870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.127761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.138406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.139315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.139670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.141225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.141225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:43.142603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.146797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.181066 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1026/2032] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:43.182152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.204594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76258acb-df84-437b-88d8-9e6af34af4af admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.211517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.212292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.212639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.212925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.213624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.215811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.234277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.236059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.245743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1007/2033] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:43.256039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1027/2034] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:43 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:43.256772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.268950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104947f7-d893-4c8a-a03e-e4a1b9d5977b service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.316119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:43.337762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.338455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.338746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.339127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.339967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.342113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.374160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1008/2035] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:43 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:43.379312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.394089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7e743-4406-4e63-8b1f-63bae74c0aef admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.400138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.400795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.401083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.401362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.402102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.403785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.419130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1028/2036] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:43.427259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.442395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcba111-d484-49fc-bb9a-b7dcd9bd1e89 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.448323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.449249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.449607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.449987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.450789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.452849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.464440 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1009/2037] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:43.469678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.481330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885f6d-621d-4d87-8c50-fdfd6b8ac10c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.486854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.488106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.488402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.488682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] PATH_INFO: `/v3/projects/6c5b285a04564d5d917a4566bec984b2/users/1e4d2ec6d0014c99a818e1000869b38c/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.502505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.504737 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.513979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5b285a04564d5d917a4566bec984b2, user_id=1e4d2ec6d0014c99a818e1000869b38c, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.517284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd465-5b5e-493a-a1c4-430954228893 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.558363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1029/2038] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:43 2022] PUT /identity/v3/projects/6c5b285a04564d5d917a4566bec984b2/users/1e4d2ec6d0014c99a818e1000869b38c/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:43.564315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.564888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.565310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.588351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:43.589781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.616701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83cc71-815f-4489-8d6f-23c0d7fafe12 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.677544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1030/2039] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:43.686108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.687311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.687602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.687707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:43.687954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.689455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.689866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:43.737338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:43.756221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a5cf-4050-4547-b3c1-3502a4798bc2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:43.760398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:43.761175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:43.761538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:43.761886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:43.765783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.767290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2017e-e05d-4e58-b68c-c2f535cf2e33 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.772065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:43.773614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:43.777973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1010/2040] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:43.784652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1031/2041] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:42:44.680968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd0c1e-0581-4512-ab2d-b98869ef4586 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:44.681406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd0c1e-0581-4512-ab2d-b98869ef4586 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:44.681773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd0c1e-0581-4512-ab2d-b98869ef4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:44.711839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-62cd0c1e-0581-4512-ab2d-b98869ef4586 None None] MFA Rules not processed for user `1e4d2ec6d0014c99a818e1000869b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:44.713087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-62cd0c1e-0581-4512-ab2d-b98869ef4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:44.734603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:44.805492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1011/2042] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:44.814152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:44.825710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773467-a67a-4440-a19d-9de02fff28e5 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:44.829285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:44.829944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:44.830227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:44.830641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:44.839752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:44.841315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:44.850581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1032/2043] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:46.211191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:46.211685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:46.211685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:46.231648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] MFA Rules not processed for user `1e4d2ec6d0014c99a818e1000869b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:46.232911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:46.241754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1012/2044] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:49.258087 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:49.268331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca51d4e-45f8-44ad-987d-8a9333f678e4 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:49.272442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:49.273204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:49.273491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:49.273979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:49.276102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:49.357834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:49.360680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf587c8f-aef6-48c8-bf8a-d782100edf01 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:49.378879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1033/2045] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:51.104101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:51.106054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:51.122584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d5cd-3cd3-497c-9072-5e78f1577a0e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:51.128766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:51.129771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:51.130075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:51.130335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] PATH_INFO: `/v3/users/0da1a254911942dab58691f79b3151a1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:51.131690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da1a254911942dab58691f79b3151a1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:51.133684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:51.188248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da1a254911942dab58691f79b3151a1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.196883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da1a254911942dab58691f79b3151a1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.206236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da1a254911942dab58691f79b3151a1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.206972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da1a254911942dab58691f79b3151a1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.207819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da1a254911942dab58691f79b3151a1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.230443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1013/2046] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:51 2022] DELETE /identity/v3/users/0da1a254911942dab58691f79b3151a1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:51.235572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:51.236037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:51.244288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:51.256913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:51.258125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:51.319137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1034/2047] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:51.330415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:51.332061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:51.416156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa285daa-42e8-4aa7-80d9-fd7edd4e3b97 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:51.431929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:51.434446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:51.434885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:51.435489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:51.449255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:51.453638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:51.477810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1014/2048] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:51.877263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:51.879323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:51.897677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d760c-c117-4939-82c3-63a6521ff89b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:51.902499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:51.904975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:51.904975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:51.905166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] PATH_INFO: `/v3/projects/68f1b01f384b4970bfeca47ff8629136` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:51.906240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f1b01f384b4970bfeca47ff8629136)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:51.908289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:51.948004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:51.949684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:51.962094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f1b01f384b4970bfeca47ff8629136', 'request_id': 'req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.963421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d667-4d2b-4b48-a406-d1dae55848cd service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:51.970039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f1b01f384b4970bfeca47ff8629136', 'request_id': 'req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.981086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f1b01f384b4970bfeca47ff8629136', 'request_id': 'req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.982927 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:51.996505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] The token cache is being invalidate because project 68f1b01f384b4970bfeca47ff8629136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:51.997188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.997980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:51.998215 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:52.000093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1035/2049] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:51 2022] DELETE /identity/v3/projects/68f1b01f384b4970bfeca47ff8629136 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:52.003995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:52.005578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.040379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:52.041388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.041569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.041853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] PATH_INFO: `/v3/users/27816a06229541f88bdfca45f2c4c232` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.043814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27816a06229541f88bdfca45f2c4c232)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:52.045790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:52.099422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-30b9-45b3-a52b-4f49fcc2583b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:52.107717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:52.109411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.110150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.110441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] PATH_INFO: `/v3/users/69f054c5dea44385bba6199c462f3748` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.111649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27816a06229541f88bdfca45f2c4c232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.112103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f054c5dea44385bba6199c462f3748)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:52.113858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:52.120822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27816a06229541f88bdfca45f2c4c232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.132960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27816a06229541f88bdfca45f2c4c232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.133369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27816a06229541f88bdfca45f2c4c232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.133979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17b363e3-4685-452f-9e5f-a67bcf9e5141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27816a06229541f88bdfca45f2c4c232'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.161922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1015/2050] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:51 2022] DELETE /identity/v3/users/27816a06229541f88bdfca45f2c4c232 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:52.168807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.169247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.169527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.195282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f054c5dea44385bba6199c462f3748'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.205494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:52.206720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.211817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f054c5dea44385bba6199c462f3748'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.212791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f054c5dea44385bba6199c462f3748'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.226356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f054c5dea44385bba6199c462f3748'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.238172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f054c5dea44385bba6199c462f3748'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.251007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1036/2051] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:52 2022] DELETE /identity/v3/users/69f054c5dea44385bba6199c462f3748 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:52.333848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1016/2052] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:52.358710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:52.361218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.437542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481944dd-3e0d-4897-97d5-9695c5eee4f1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:52.441359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:52.442030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.442309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.442582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.465090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:52.532104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:52.533939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92fb63-5752-44d4-a38d-5ee18acb7760 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:52.556664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1037/2053] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:52.651767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:52.653857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.676751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c3a43-0dfa-41fd-a291-9921ac23b260 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:52.682255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:52.683332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.683643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.683995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] PATH_INFO: `/v3/projects/4e80cd6012e54426a18e8ce8b731eba4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.685290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e80cd6012e54426a18e8ce8b731eba4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:52.687277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:52.747870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:52.769023 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e80cd6012e54426a18e8ce8b731eba4', 'request_id': 'req-142a8e9c-5da6-4f73-823b-e93912d0a7ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.792026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e80cd6012e54426a18e8ce8b731eba4', 'request_id': 'req-142a8e9c-5da6-4f73-823b-e93912d0a7ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.797510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e80cd6012e54426a18e8ce8b731eba4', 'request_id': 'req-142a8e9c-5da6-4f73-823b-e93912d0a7ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.814090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] The token cache is being invalidate because project 4e80cd6012e54426a18e8ce8b731eba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:52.814090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.814942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:52.819365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1017/2054] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:52 2022] DELETE /identity/v3/projects/4e80cd6012e54426a18e8ce8b731eba4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:52.823108 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b306f-7190-48e4-8391-74097c498c57 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.823772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b306f-7190-48e4-8391-74097c498c57 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.823772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b306f-7190-48e4-8391-74097c498c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.855237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b72b306f-7190-48e4-8391-74097c498c57 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:52.857638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b72b306f-7190-48e4-8391-74097c498c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.950828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:52.957150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1038/2055] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:52.957336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.959392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.959779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.960172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.971745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a8e9c-5da6-4f73-823b-e93912d0a7ed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:52.976876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:52.977850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:52.978319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:52.979095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] PATH_INFO: `/v3/projects/cff9b2b9836b4af09eb6649d24b51d90` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:52.980457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff9b2b9836b4af09eb6649d24b51d90)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:52.982904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:52.985812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:52.987166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:52.997512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1039/2056] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:53.001875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.011960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0cf13-7a28-46ff-9435-c11bdf3a490e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.017585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.018394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.018813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.019154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.019947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:53.020333 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.022047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.035010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1040/2057] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:42:53.039271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.039785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff9b2b9836b4af09eb6649d24b51d90', 'request_id': 'req-ddcec064-27d0-46f2-8dc5-c62e6e84000b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.050328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cad88e-f631-4fad-b5a2-c7c74ae491c9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.051521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff9b2b9836b4af09eb6649d24b51d90', 'request_id': 'req-ddcec064-27d0-46f2-8dc5-c62e6e84000b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.061078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff9b2b9836b4af09eb6649d24b51d90', 'request_id': 'req-ddcec064-27d0-46f2-8dc5-c62e6e84000b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.076176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] The token cache is being invalidate because project cff9b2b9836b4af09eb6649d24b51d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:53.076468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.077177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.079201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1018/2058] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:52 2022] DELETE /identity/v3/projects/cff9b2b9836b4af09eb6649d24b51d90 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:53.083663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.085240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:53.116614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.117393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.117751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.118097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.118859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.120939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.154348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1041/2059] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:53 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:53.158384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.160188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:53.160785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcec064-27d0-46f2-8dc5-c62e6e84000b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.189756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:53.233696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961439-cf36-46ce-8fa6-28deb0a791e0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.239850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.240858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.241212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.241589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.242456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.246294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.259524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.260645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.261141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.261353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] PATH_INFO: `/v3/users/7d3818b0b6e04850b9673f6e0defd5e8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.262807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3818b0b6e04850b9673f6e0defd5e8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.265193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.268076 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:53.271417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1042/2060] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:53 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:53.275067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.285913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae909d1-b058-4139-806d-7c61f5fbfb69 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.292356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.293727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.294065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.294404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.295248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.297635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.313091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1043/2061] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:42:53.314267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.334744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94f03-f4f0-4527-996e-56fb74d8bcdd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.341794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.342509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.342816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.343062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.344466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.345761 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.375205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1044/2062] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:53.381127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3818b0b6e04850b9673f6e0defd5e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.384439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.403834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3818b0b6e04850b9673f6e0defd5e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.410211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7640-8e86-4a13-9d28-2bc3bc856df8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.421638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3818b0b6e04850b9673f6e0defd5e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.422148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.423607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3818b0b6e04850b9673f6e0defd5e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.424158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.425187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-64773871-0c5f-433e-a6d9-6c9a7ef6131b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3818b0b6e04850b9673f6e0defd5e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:53.425994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.426289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] PATH_INFO: `/v3/projects/e9c2f74e24b94a42b73d2af159081a71/users/310d05b9130a456f88a486ac2f44b12a/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.457544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c2f74e24b94a42b73d2af159081a71, user_id=310d05b9130a456f88a486ac2f44b12a, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.460807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.483636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1019/2063] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:53 2022] DELETE /identity/v3/users/7d3818b0b6e04850b9673f6e0defd5e8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:53.487318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1045/2064] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:53 2022] PUT /identity/v3/projects/e9c2f74e24b94a42b73d2af159081a71/users/310d05b9130a456f88a486ac2f44b12a/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:53.499726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.500136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.500483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.537365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] MFA Rules not processed for user `310d05b9130a456f88a486ac2f44b12a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:53.541052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:53.654513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1020/2065] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:53.657211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:53.659876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:53.787919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b9ad7-9311-452b-94f3-ebd4cd4ef7e3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:53.916882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:53.918351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:53.918744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:53.919407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:53.922302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:53.925119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:53.969427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:53.984065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1046/2066] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:53 2022] POST /identity/v3/projects => generated 387 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:53.987843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.009241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92331718-c8c7-43d6-adc1-1e6b6aa2877f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.112378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.114621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:54.120384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.122273 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.122873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.123284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.124359 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.128810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.137557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196492a-7737-41b3-90d8-28b351b1e7ed admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.156868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.157792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.158319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.158533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] PATH_INFO: `/v3/projects/983f87973fb94a138dbd3ee463cb2225` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.159602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:54.160632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983f87973fb94a138dbd3ee463cb2225)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.162718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.171290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1021/2067] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:53 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:54.171986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.187130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df3c6-069e-42b2-a5b7-ce234a01c1c7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.192243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.193090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.193701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.194172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.195033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.197401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.209259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1022/2068] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:54.213675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.224792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6f72b-1781-408e-9d3e-db2e8f2b0092 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.230912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.232431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.232823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.233192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] PATH_INFO: `/v3/projects/edbbb1db92de401c86272fdba18c77d6/users/ff3401f258dd47f6a6492863b26cb6c1/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.244550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983f87973fb94a138dbd3ee463cb2225', 'request_id': 'req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.252121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbbb1db92de401c86272fdba18c77d6, user_id=ff3401f258dd47f6a6492863b26cb6c1, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.253281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983f87973fb94a138dbd3ee463cb2225', 'request_id': 'req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.255516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.262482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983f87973fb94a138dbd3ee463cb2225', 'request_id': 'req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.274347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] The token cache is being invalidate because project 983f87973fb94a138dbd3ee463cb2225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:54.274635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.275270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1023/2069] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:54 2022] PUT /identity/v3/projects/edbbb1db92de401c86272fdba18c77d6/users/ff3401f258dd47f6a6492863b26cb6c1/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:54.278363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.279299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.281110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1047/2070] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:54 2022] DELETE /identity/v3/projects/983f87973fb94a138dbd3ee463cb2225 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:54.281886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:54.284812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.286233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:54.363729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5742e-83d5-4ed2-99dd-979b8fdf73d3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.366706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703697-3a77-45fd-a1ef-9f139f37cfdc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.369315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.371500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.371775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.372059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] PATH_INFO: `/v3/users/da324f3736fb4f80ae45e0b1f1b3dfdc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.372679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.373315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da324f3736fb4f80ae45e0b1f1b3dfdc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.373735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.374149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.374556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.375492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.375662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.377659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.391926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1024/2071] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:54.393394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.405865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc5733-11ba-4bbe-a9fd-a79ea73d1c00 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.413834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.414710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.415029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.415310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] PATH_INFO: `/v3/projects/edbbb1db92de401c86272fdba18c77d6/users/ff3401f258dd47f6a6492863b26cb6c1/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.417900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbbb1db92de401c86272fdba18c77d6, user_id=ff3401f258dd47f6a6492863b26cb6c1, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.421777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853dfca-fbcd-4d8f-bac3-5122cd5109c5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.439162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1025/2072] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:54 2022] PUT /identity/v3/projects/edbbb1db92de401c86272fdba18c77d6/users/ff3401f258dd47f6a6492863b26cb6c1/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:54.448684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.448990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da324f3736fb4f80ae45e0b1f1b3dfdc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.449132 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.449356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.463883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da324f3736fb4f80ae45e0b1f1b3dfdc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.464452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da324f3736fb4f80ae45e0b1f1b3dfdc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.477731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da324f3736fb4f80ae45e0b1f1b3dfdc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.487753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] MFA Rules not processed for user `ff3401f258dd47f6a6492863b26cb6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:54.488973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:54.489899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da324f3736fb4f80ae45e0b1f1b3dfdc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:54.499830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1048/2073] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:54 2022] DELETE /identity/v3/users/da324f3736fb4f80ae45e0b1f1b3dfdc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:54.581756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1026/2074] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:54.589515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:54.592219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:54.599433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:54.685132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc1a1f-86e2-4b84-a0b6-e888f7fb6bb6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:54.688776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:54.689451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:54.689881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:54.690211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:54.769100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:54.770721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45438a9f-6522-4b51-8101-47ac05575a7e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:54.781405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1049/2075] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:55.064956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.066684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.139355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:55.173196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0b4c9-ac0e-4ea5-882a-2ec7b8b9c1c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.241691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.242664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.243022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.243540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] PATH_INFO: `/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.244907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8ed9945d9b48c9826222a3a29ea14e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.248224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.308267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8ed9945d9b48c9826222a3a29ea14e', 'request_id': 'req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.317656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8ed9945d9b48c9826222a3a29ea14e', 'request_id': 'req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.328262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8ed9945d9b48c9826222a3a29ea14e', 'request_id': 'req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.345043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] The token cache is being invalidate because project 9f8ed9945d9b48c9826222a3a29ea14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:55.345333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.346051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8ebeb51-60b3-48ce-856d-a15127a3f3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.347872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1027/2076] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:55 2022] DELETE /identity/v3/projects/9f8ed9945d9b48c9826222a3a29ea14e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:55.355520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.356419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.356419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.375219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:55.376441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.455415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1050/2077] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:55.466246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.466535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.466957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.485376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.487330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.488027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:55.489279 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.500637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19beb761-a242-4be5-a1d4-21b7ec935e22 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.500975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1028/2078] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:55.505153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.509661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.510822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.512677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.512677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] PATH_INFO: `/v3/users/310d05b9130a456f88a486ac2f44b12a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.514380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] RBAC: Authorizing `identity:delete_user(user_id=310d05b9130a456f88a486ac2f44b12a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.516384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.517817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686afb-4b7c-44c7-b816-69aa0dc89fb8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.523434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.524074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.524360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.524630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.525418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:55.525730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.527498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.539879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1029/2079] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:42:55.544352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.555342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc9ebc-0ee2-4788-9697-568d3493cc47 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.562973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.563965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.564411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.564785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.565656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.568009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.574676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '310d05b9130a456f88a486ac2f44b12a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.575815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:55.590686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '310d05b9130a456f88a486ac2f44b12a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.591571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '310d05b9130a456f88a486ac2f44b12a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.596367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1030/2080] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:55.600684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '310d05b9130a456f88a486ac2f44b12a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.600875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.608959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94c7739c-83cf-4909-820a-395e2fc8074c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '310d05b9130a456f88a486ac2f44b12a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:55.615276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1051/2081] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:55 2022] DELETE /identity/v3/users/310d05b9130a456f88a486ac2f44b12a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:55.619392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.619754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.620101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.638516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:55.639785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.640244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:55.676793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68de10-dff0-431c-b08a-93c5cafed4ba admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.710428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1052/2082] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:55.722145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.723582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.740143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.740885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.747115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.747115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.747115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.747115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.776088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1031/2083] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:55.782540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.785238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:55.800241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216e238-e48b-4748-bc46-0725bf5dec30 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.805242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670049f6-ec90-4b27-ad36-75939f319d61 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.806517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.807436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.807928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.808402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.809387 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.810842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.811311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.815915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.816271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.816554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.834642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.836527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.839212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1032/2084] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:55.843677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.849999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1053/2085] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:55.856421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569d822-98c5-47ac-986d-88c04632afd9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.862681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.863428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.863748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.864032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.864822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.866510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd8f3e-9a15-4f7e-8408-5c0a40521aaa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.879552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1033/2086] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:55.883834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:55.894471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f708d5-ee0e-4372-abbc-23931458e839 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:55.900074 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:55.901062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.901435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.901856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] PATH_INFO: `/v3/projects/0953e749bdff4b398bb552c4ac09c48d/users/892b204b7eba438a983aadb4289edcda/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:55.917294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0953e749bdff4b398bb552c4ac09c48d, user_id=892b204b7eba438a983aadb4289edcda, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:55.920084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:55.935488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1054/2087] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:55 2022] PUT /identity/v3/projects/0953e749bdff4b398bb552c4ac09c48d/users/892b204b7eba438a983aadb4289edcda/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:42:55.946780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:55.947094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:55.947379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:56.003997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] MFA Rules not processed for user `892b204b7eba438a983aadb4289edcda`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:56.005419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:56.085582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1034/2088] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:56.098832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:56.104239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:56.178384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:56.210043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1460b8c-ed2f-49c4-bfa6-1865cf83328b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:56.213840 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:56.214697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:56.214929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:56.217171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:56.225136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:56.226790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:56.236671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1055/2089] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:56.250893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:56.252748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:56.302667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:56.364682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71933a67-738f-4bd5-8dff-ed62cda7dab7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:56.372197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:56.373255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:56.373537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:56.373811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] PATH_INFO: `/v3/projects/e9c2f74e24b94a42b73d2af159081a71` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:56.375134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c2f74e24b94a42b73d2af159081a71)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:56.377276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:56.477332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c2f74e24b94a42b73d2af159081a71', 'request_id': 'req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.498520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c2f74e24b94a42b73d2af159081a71', 'request_id': 'req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.511504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c2f74e24b94a42b73d2af159081a71', 'request_id': 'req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.545067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] The token cache is being invalidate because project e9c2f74e24b94a42b73d2af159081a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:56.545882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.548547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.551506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1035/2090] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:56 2022] DELETE /identity/v3/projects/e9c2f74e24b94a42b73d2af159081a71 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:56.556768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:56.577398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab18c2-5a5f-4442-bf72-f6b12439f1d6 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:56.706410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:56.707402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:56.707689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:56.707967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] PATH_INFO: `/v3/users/ff3401f258dd47f6a6492863b26cb6c1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:56.711802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3401f258dd47f6a6492863b26cb6c1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:56.714004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:56.801444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3401f258dd47f6a6492863b26cb6c1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.816353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3401f258dd47f6a6492863b26cb6c1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.817112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3401f258dd47f6a6492863b26cb6c1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.826106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3401f258dd47f6a6492863b26cb6c1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.833306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3401f258dd47f6a6492863b26cb6c1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:56.844998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1056/2091] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:56 2022] DELETE /identity/v3/users/ff3401f258dd47f6a6492863b26cb6c1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:57.674094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:57.675987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:57.690079 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b2fcf-faf7-41fe-a231-f4785f13cf1e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:57.827014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:57.827947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:57.829668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:57.829968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] PATH_INFO: `/v3/projects/edbbb1db92de401c86272fdba18c77d6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:57.833882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edbbb1db92de401c86272fdba18c77d6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:57.838949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:57.956811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edbbb1db92de401c86272fdba18c77d6', 'request_id': 'req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:57.967968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edbbb1db92de401c86272fdba18c77d6', 'request_id': 'req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:57.983001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edbbb1db92de401c86272fdba18c77d6', 'request_id': 'req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:57.984624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:58.009055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] The token cache is being invalidate because project edbbb1db92de401c86272fdba18c77d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:58.009521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.012471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9d63ec1-fcba-4136-bfb3-67329ba24ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.015386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1036/2092] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:57 2022] DELETE /identity/v3/projects/edbbb1db92de401c86272fdba18c77d6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:58.183425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:58.201629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:58.257499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714496e-0342-4ed4-8e0f-e14d381a2016 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:58.390631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:58.391694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:58.392244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:58.392636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] PATH_INFO: `/v3/users/1b8573a17f1b449a8040efb4e7918381` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:58.394061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8573a17f1b449a8040efb4e7918381)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:58.396561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:58.404093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:58.508198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8573a17f1b449a8040efb4e7918381'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.543844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8573a17f1b449a8040efb4e7918381'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.546259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8573a17f1b449a8040efb4e7918381'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.572212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8573a17f1b449a8040efb4e7918381'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.590969 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8573a17f1b449a8040efb4e7918381'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:58.614875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1057/2093] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:58 2022] DELETE /identity/v3/users/1b8573a17f1b449a8040efb4e7918381 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:58.619487 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:58.621647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:58.622098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:58.649131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:58.651022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:58.751678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1037/2094] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:58.766195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:58.767716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:58.862625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85e8ae-60c8-4176-b869-bed3c479816e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:58.868853 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:58.869534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:58.869917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:58.870267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:58.882520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:58.884215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e774b6-ac41-4539-928e-4ea278480571 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:58.901335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1058/2095] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:42:59.200292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.202133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:59.221452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070dcdfd-9b1c-427f-8da9-31c739e5650a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.228308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.228829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.229196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.229650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] PATH_INFO: `/v3/projects/2334b95fd02b47b1b354dc8c7fed8ec4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.232107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2334b95fd02b47b1b354dc8c7fed8ec4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.234191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.289193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2334b95fd02b47b1b354dc8c7fed8ec4', 'request_id': 'req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:59.298752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2334b95fd02b47b1b354dc8c7fed8ec4', 'request_id': 'req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:59.304926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2334b95fd02b47b1b354dc8c7fed8ec4', 'request_id': 'req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:59.317929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] The token cache is being invalidate because project 2334b95fd02b47b1b354dc8c7fed8ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:42:59.318226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:59.318976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f8d5a9da-83fe-44dd-b3d1-e40e441d49bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:42:59.320691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1038/2096] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:42:59 2022] DELETE /identity/v3/projects/2334b95fd02b47b1b354dc8c7fed8ec4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:59.325944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.326470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.326470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.344818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:59.346037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:59.409984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1059/2097] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:59.418354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.418658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.418940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.437972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:59.439222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:59.447790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1039/2098] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:42:59.452294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.461833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba346-ceae-4f50-a950-1d7f353400e7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.466754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.468165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.468452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.468731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.469757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:42:59.470040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.471582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.484548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1060/2099] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:42:59.489084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.500040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907f58-f91c-4071-a967-7392596a7d13 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.505033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.505716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.505997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.506270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.506987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.508959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.537846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1040/2100] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:42:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:42:59.542023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.552920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfef24-e209-45f6-af5d-83ac74c22993 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.566531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:59.617731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.618449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.618774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.619107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.619850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.621910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.647393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1061/2101] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:42:59 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:42:59.648892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.679939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca594cc-1ac1-46af-b74b-e317a61799d4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.688465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.689177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.689468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.689753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.690831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.692615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.710569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1041/2102] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:59.720366 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.733651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993817-fcfe-4be2-9aee-cfca1e2a0e37 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.738645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.739306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.739652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.740007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.749275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.751129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aceb82-696f-4a0a-9d8f-ee1fac342f8a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.771155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1062/2103] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:42:59.774811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.787603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a9911-ddd7-4879-8670-e7e116e8dc77 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:42:59.793718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:42:59.794791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.795251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.795657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] PATH_INFO: `/v3/projects/1e987dd9cf634d269c852374bf501a87/users/5e45fa7f9a5f407db80f1726c9df0c8d/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.821877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e987dd9cf634d269c852374bf501a87, user_id=5e45fa7f9a5f407db80f1726c9df0c8d, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:42:59.826875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:42:59.858963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1042/2104] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:42:59 2022] PUT /identity/v3/projects/1e987dd9cf634d269c852374bf501a87/users/5e45fa7f9a5f407db80f1726c9df0c8d/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:42:59.860493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:42:59.861156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:42:59.861451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:42:59.890832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] MFA Rules not processed for user `5e45fa7f9a5f407db80f1726c9df0c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:42:59.892253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:59.937127 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.938951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:42:59.953969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:42:59.983667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1063/2105] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:42:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:42:59.994384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:42:59.995905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.030034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4f7e-8809-4d16-a1a6-5762eef6c50c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.034812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.035737 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.036021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.036296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] PATH_INFO: `/v3/users/892b204b7eba438a983aadb4289edcda` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.037511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=892b204b7eba438a983aadb4289edcda)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.039571 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:00.059449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9f5d-1a4f-42be-8c68-b4ba5123e3c0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.063958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.064632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.064919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.065268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.074918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.076423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0657b-7cb7-412f-a3a5-12555a73a5ae service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:00.086610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1064/2106] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:42:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:43:00.098690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892b204b7eba438a983aadb4289edcda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.107109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892b204b7eba438a983aadb4289edcda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.116567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892b204b7eba438a983aadb4289edcda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.117290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892b204b7eba438a983aadb4289edcda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.117907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892b204b7eba438a983aadb4289edcda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.137860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1043/2107] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:42:59 2022] DELETE /identity/v3/users/892b204b7eba438a983aadb4289edcda => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:00.143226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.143582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.143920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.167117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:00.168252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.188407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:00.229000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1065/2108] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:43:00.240229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:00.242095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.305603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f1dce-e894-4f13-8992-3e21e7ed9666 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.310130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.310797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.311105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.311391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.321533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.323221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9378a5d-738e-49b0-9370-03db3319ecc3 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:00.334213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1044/2109] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:00.634354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:00.636748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.654433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d06238-970e-4f2a-b839-a207dc0c9b48 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.663671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.664564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.664841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.665110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] PATH_INFO: `/v3/projects/0953e749bdff4b398bb552c4ac09c48d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.666428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0953e749bdff4b398bb552c4ac09c48d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.668912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:00.717950 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0953e749bdff4b398bb552c4ac09c48d', 'request_id': 'req-34a2a98a-a61a-4d88-bddc-26655f1a8d79'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.724317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0953e749bdff4b398bb552c4ac09c48d', 'request_id': 'req-34a2a98a-a61a-4d88-bddc-26655f1a8d79'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.734110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0953e749bdff4b398bb552c4ac09c48d', 'request_id': 'req-34a2a98a-a61a-4d88-bddc-26655f1a8d79'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.748408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] The token cache is being invalidate because project 0953e749bdff4b398bb552c4ac09c48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:00.748696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.750206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-34a2a98a-a61a-4d88-bddc-26655f1a8d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:00.751725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1066/2110] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:00 2022] DELETE /identity/v3/projects/0953e749bdff4b398bb552c4ac09c48d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:00.758635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.759025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.759429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.779065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:00.780307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.815661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:00.848209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1045/2111] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:00.857889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.858184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.858466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.878505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:00.879791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:00.892976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1067/2112] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:00.899017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:00.913521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53625362-4d91-4b3d-851a-79036e5e27db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.921386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.921997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.922269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.923190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.923934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:00.924244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.925873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:00.938825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1046/2113] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:43:00.943319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:00.958549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae020a-98ce-4913-831d-aae92a44383e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:00.968156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:00.969294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:00.969803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:00.970307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:00.971592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:00.974925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.024664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1068/2114] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:00 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:01.028981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.040034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee8192-333f-4206-bf55-2e2bd7865d30 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.103549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.104394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.104840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.104998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.105681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.108028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.137134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1047/2115] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:01 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:01.140980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.153700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086645e2-33da-47c5-a3e7-131db43c5ea9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.160263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.161355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.161740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.162118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.163344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.165144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.177137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1069/2116] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:01.181111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.199529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012df529-3ce5-47d6-8527-99b24d3cc4ca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.206937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.206937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.206937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.206937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.207596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.209298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464401dc-cde1-4198-bf00-51ba8248e45e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.221976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1048/2117] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:01.226715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.237949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7329-95c6-4a57-8b3d-6d9dfe2d8340 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.243532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.244464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.244817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.245006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] PATH_INFO: `/v3/projects/47858c14999b4cc89a3c06d73a28e9a0/users/ff931e661b1947dca2df8bff6a338909/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.262095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=47858c14999b4cc89a3c06d73a28e9a0, user_id=ff931e661b1947dca2df8bff6a338909, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.264992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.292701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1070/2118] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:01 2022] PUT /identity/v3/projects/47858c14999b4cc89a3c06d73a28e9a0/users/ff931e661b1947dca2df8bff6a338909/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:01.295204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.295632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.296012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.327748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] MFA Rules not processed for user `ff931e661b1947dca2df8bff6a338909`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:01.329255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:01.402569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1049/2119] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:01.407790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.461207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:01.481735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd4133-b86a-402e-a787-7d8c866de8fe admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.487716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.488526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.488819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.489130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.489782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.492905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.526356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1071/2120] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:01 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:01.530641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.545026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d6b1-c787-4f83-ad13-263d13bfed34 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.597390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:01.639742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.640500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.640927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.641350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.642313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.644942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.673338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1050/2121] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:01 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:01.677698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.689832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ee6df-b89e-460e-8ccd-e6a4866b12df admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.696204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.697043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.697414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.697781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.698661 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.700417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.713403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1072/2122] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:01.717687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.729810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc91ca-1253-4424-a5da-b7a890587f69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.736230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.736987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.737277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.737562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.738404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.740636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0505d-e85c-4262-86d9-ab4fd893954d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.762384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1051/2123] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:01.765943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:01.781138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812ad11-10de-4700-9df4-500b2bc7dcc7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:01.787057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:01.788126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.788506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.788886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] PATH_INFO: `/v3/projects/5b065af19fa043338bf0e47f4427fcb3/users/159059b175dd439dab5d3ff948aac84b/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.808690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b065af19fa043338bf0e47f4427fcb3, user_id=159059b175dd439dab5d3ff948aac84b, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:01.812018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:01.830657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1073/2124] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:01 2022] PUT /identity/v3/projects/5b065af19fa043338bf0e47f4427fcb3/users/159059b175dd439dab5d3ff948aac84b/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:01.840886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:01.841365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:01.841624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:01.914292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] MFA Rules not processed for user `159059b175dd439dab5d3ff948aac84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:01.915585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:02.033876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1052/2125] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:02.038526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.108455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68511dd-b9e8-41dd-8b8c-ff2a4eaa437a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.112976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.113648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.113957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.114257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.114924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.116870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.147327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1074/2126] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:02 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:02.148967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.159943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89de4ee-53c9-4f1d-8a3e-9180677d2e44 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.253575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.254461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.254838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.255287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.256091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.258946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.283901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1053/2127] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:02 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:02.288065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.303547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec95de4-c1e7-4b47-ae1a-e1f7c0de7404 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.308776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.309585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.309958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.310475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.311363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.313109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.325468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1075/2128] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:02.330792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.342318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181034-0b3e-41bd-8e85-b4d60423d031 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.347888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.348824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.349064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.349354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] PATH_INFO: `/v3/projects/702402c1ae744f898c1e172358154e41/users/783f9a504f7a41be9484d3d2bd42797b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.366796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702402c1ae744f898c1e172358154e41, user_id=783f9a504f7a41be9484d3d2bd42797b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.369562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.383495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1054/2129] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:02 2022] PUT /identity/v3/projects/702402c1ae744f898c1e172358154e41/users/783f9a504f7a41be9484d3d2bd42797b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:02.387918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.397691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a119-e831-4b8c-bbc9-e73e3e579aa7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.456783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.457525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.457803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.458076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.458805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.460534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700d8ef-9803-45b7-9249-695466909aa9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.471726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1076/2130] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:02.475747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.484208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:02.488837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c16608-0a6d-4e60-9730-ccd0c994ab76 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.495462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.496400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.496744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.497373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] PATH_INFO: `/v3/projects/702402c1ae744f898c1e172358154e41/users/783f9a504f7a41be9484d3d2bd42797b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.499607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702402c1ae744f898c1e172358154e41, user_id=783f9a504f7a41be9484d3d2bd42797b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.502403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.514622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1055/2131] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:02 2022] PUT /identity/v3/projects/702402c1ae744f898c1e172358154e41/users/783f9a504f7a41be9484d3d2bd42797b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:02.526200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.526562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.526845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.539480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:02.556705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] MFA Rules not processed for user `783f9a504f7a41be9484d3d2bd42797b`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:02.558095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:02.628941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1077/2132] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:02.640825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:02.642703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:02.702117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850930b5-b92f-4734-a256-9191b51f7018 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:02.706005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:02.706644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:02.707357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:02.707357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:02.771394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:02.772973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:02.782758 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1056/2133] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:04.304220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:04.315142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4e2b-930b-4073-baf6-6fe896a90a5a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:04.319800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:04.320518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:04.320796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:04.321069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:04.383269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:04.384718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f1d3e-14c3-4f7f-b1e5-e87c7cc243f3 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:04.394037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1078/2134] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:04.987873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:04.997644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0583d-9c61-46f5-8829-1000713e7830 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:05.004011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:05.005019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:05.005458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:05.005886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:05.016461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:05.018069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c79090-b452-4a90-a5d9-bad6a43f52d0 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:05.028894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1057/2135] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:08.754573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:08.755170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:08.755265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:08.756774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1079/2136] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:43:08.767179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:08.767724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:08.767724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:08.789680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:08.790910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:08.867545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:08.873442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:08.892857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1058/2137] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:43:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:08.905708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:08.907167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:08.921601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037da0e3-c302-45ee-9df5-364cc7726056 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:08.927613 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:08.928430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:08.928817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:08.929169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:08.939048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:08.940710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:08.951969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1059/2138] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:43:08.955056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f144-0fbb-4db7-b590-1ace477cea27 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:08.960080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:08.961086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:08.961395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:08.961675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] PATH_INFO: `/v3/users/f1722779f6b74af59fa05a2b63890b2d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:08.962889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1722779f6b74af59fa05a2b63890b2d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:08.965203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.018404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:09.038354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1722779f6b74af59fa05a2b63890b2d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.049538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.052516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1722779f6b74af59fa05a2b63890b2d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.053247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1722779f6b74af59fa05a2b63890b2d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.061916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec6ffc-a2a5-4001-a12d-b41fdc845b7e service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.063144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1722779f6b74af59fa05a2b63890b2d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.067217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.068310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.068591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.069039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] PATH_INFO: `/v3/users/ff931e661b1947dca2df8bff6a338909` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.070366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff931e661b1947dca2df8bff6a338909)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.071670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4dfc18c-e741-41a8-a3e0-fae98eaf1ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1722779f6b74af59fa05a2b63890b2d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.072485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.079748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1080/2139] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:08 2022] DELETE /identity/v3/users/f1722779f6b74af59fa05a2b63890b2d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:09.100777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:09.175620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff931e661b1947dca2df8bff6a338909'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.209678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff931e661b1947dca2df8bff6a338909'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.221684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff931e661b1947dca2df8bff6a338909'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.222492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff931e661b1947dca2df8bff6a338909'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.223481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff931e661b1947dca2df8bff6a338909'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.248419 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1060/2140] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:09 2022] DELETE /identity/v3/users/ff931e661b1947dca2df8bff6a338909 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:09.252554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.252845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.253124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.274979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:09.276322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.338721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.342402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.404587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1081/2141] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:09.415515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.417759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.440397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe444-4c1b-4e29-89de-86e9ef9fb395 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.445146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.446098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.446451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.447016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] PATH_INFO: `/v3/projects/4290bd187dc948bc9f1dc5a8cc272f0b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.448306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4290bd187dc948bc9f1dc5a8cc272f0b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.453588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.513855 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0dde5-b26d-4884-b9fb-fe49b90691c4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.520884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4290bd187dc948bc9f1dc5a8cc272f0b', 'request_id': 'req-603ec487-122e-410f-af83-d37e2abe7981'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.530307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4290bd187dc948bc9f1dc5a8cc272f0b', 'request_id': 'req-603ec487-122e-410f-af83-d37e2abe7981'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.538798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4290bd187dc948bc9f1dc5a8cc272f0b', 'request_id': 'req-603ec487-122e-410f-af83-d37e2abe7981'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.552304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] The token cache is being invalidate because project 4290bd187dc948bc9f1dc5a8cc272f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:09.552793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.553653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-603ec487-122e-410f-af83-d37e2abe7981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:09.556306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1061/2142] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:09 2022] DELETE /identity/v3/projects/4290bd187dc948bc9f1dc5a8cc272f0b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:09.565515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720905-d793-4b05-8bad-55c7d739083b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.566294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720905-d793-4b05-8bad-55c7d739083b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.566874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720905-d793-4b05-8bad-55c7d739083b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.582929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.585992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.586353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.586701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.588690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.589587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-41720905-d793-4b05-8bad-55c7d739083b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:09.590764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-41720905-d793-4b05-8bad-55c7d739083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.607357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:09.617698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:09.673957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1062/2143] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:09.674396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.675933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.681609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.681897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.682170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.686765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1082/2144] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:09.704753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:09.705974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.714329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1063/2145] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:43:09.718739 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.728682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38429d28-f0bb-4633-aead-486005a86277 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.734007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.734681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.734962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.735271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.735997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:09.736326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.737937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.751163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1083/2146] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:43:09.756263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.768005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66314a06-8680-40fc-b6f2-a3f349caf137 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.773096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.773777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.774061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.774338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.775034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.777246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.820208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1064/2147] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:09 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:09.821405 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.832570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5987d-4942-4632-9502-146cb9cef894 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.904657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.905385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.905665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.905961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.906660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.908626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.931168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1084/2148] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:09 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:43:09.935683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.946489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169292d-dbc6-4f66-ada8-310bbcb568dd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.951657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.952400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.952691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.952969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.953828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.955857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.968807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1065/2149] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:09.973424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.981455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:09.983681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:09.985163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334562e-a249-4268-a98c-c6d1a4f791af admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:09.990615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:09.991325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:09.991610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:09.991876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:09.992796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:09.995961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:09.998762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a5d90-d4b2-4dc3-9fc9-73ef82f09fc0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.005125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.005913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.006199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.006481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] PATH_INFO: `/v3/projects/47858c14999b4cc89a3c06d73a28e9a0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.007717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47858c14999b4cc89a3c06d73a28e9a0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.009466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1085/2150] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:10.009680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.013965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.024142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f8cb5-937c-4d22-a030-54094c2e886e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.028912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.029807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.030091 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.030370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] PATH_INFO: `/v3/projects/b9016db2909b48ebb0e60cd353c4feeb/users/d50acc7f684a47b78771ad57ab9d91b7/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.046219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9016db2909b48ebb0e60cd353c4feeb, user_id=d50acc7f684a47b78771ad57ab9d91b7, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.050093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0e83c-f477-4794-b308-8977af1a9533 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.059524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47858c14999b4cc89a3c06d73a28e9a0', 'request_id': 'req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.065526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1086/2151] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:10 2022] PUT /identity/v3/projects/b9016db2909b48ebb0e60cd353c4feeb/users/d50acc7f684a47b78771ad57ab9d91b7/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:10.075801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47858c14999b4cc89a3c06d73a28e9a0', 'request_id': 'req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.076222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.076608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.076957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.081652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47858c14999b4cc89a3c06d73a28e9a0', 'request_id': 'req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.093805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] The token cache is being invalidate because project 47858c14999b4cc89a3c06d73a28e9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:10.094102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.094853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.096691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1066/2152] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:09 2022] DELETE /identity/v3/projects/47858c14999b4cc89a3c06d73a28e9a0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:10.100696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.102165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.105210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] MFA Rules not processed for user `d50acc7f684a47b78771ad57ab9d91b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:10.106650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.153195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:10.168059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01cbfc-d5f8-4eeb-8d6e-d40c410fdd1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.173150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:10.173475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1087/2153] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:43:10.185528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.187297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.233813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.234709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.234992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.235319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] PATH_INFO: `/v3/users/159059b175dd439dab5d3ff948aac84b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.236549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=159059b175dd439dab5d3ff948aac84b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.238486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.253949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85084b9-1e6c-4c07-956b-27fdf90e0f2d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.258574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.259213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.259502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.259780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.270921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.272734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.282723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1088/2154] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:10.288366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '159059b175dd439dab5d3ff948aac84b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.295631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '159059b175dd439dab5d3ff948aac84b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.304016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '159059b175dd439dab5d3ff948aac84b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.304856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '159059b175dd439dab5d3ff948aac84b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.305621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-12259e32-d9a2-4d4a-8f58-2221c3cd31ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '159059b175dd439dab5d3ff948aac84b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.325407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1067/2155] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:10 2022] DELETE /identity/v3/users/159059b175dd439dab5d3ff948aac84b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:10.393035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.395780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.412241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc582c-6fe6-4e98-bd03-225edefebc5f service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.501505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.502479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.502763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.503040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] PATH_INFO: `/v3/users/5e45fa7f9a5f407db80f1726c9df0c8d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.504422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e45fa7f9a5f407db80f1726c9df0c8d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.506552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.612935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e45fa7f9a5f407db80f1726c9df0c8d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.626636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e45fa7f9a5f407db80f1726c9df0c8d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.627599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e45fa7f9a5f407db80f1726c9df0c8d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.637526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e45fa7f9a5f407db80f1726c9df0c8d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.646653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e45fa7f9a5f407db80f1726c9df0c8d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.657958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1089/2156] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:10 2022] DELETE /identity/v3/users/5e45fa7f9a5f407db80f1726c9df0c8d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:10.666379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.666379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.666379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.684720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:10.686035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.766858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1068/2157] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:10.777145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.778772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.791910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:10.815874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.817500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:10.830163 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe540bbc-26be-4d3a-ad40-a91b4c6d5f06 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.835595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.836583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.837742 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.839142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] PATH_INFO: `/v3/projects/5b065af19fa043338bf0e47f4427fcb3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.840585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b065af19fa043338bf0e47f4427fcb3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.842544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.852056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec182c5c-430a-4460-9227-35efc38dab54 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:10.857258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:10.858309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:10.858601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:10.858884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:10.870067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:10.872144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:10.882244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:10.883106 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1090/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:10.909575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b065af19fa043338bf0e47f4427fcb3', 'request_id': 'req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.919018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b065af19fa043338bf0e47f4427fcb3', 'request_id': 'req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.928068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b065af19fa043338bf0e47f4427fcb3', 'request_id': 'req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.943194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] The token cache is being invalidate because project 5b065af19fa043338bf0e47f4427fcb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:10.943501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.944234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:10.946134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1069/2159] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:10 2022] DELETE /identity/v3/projects/5b065af19fa043338bf0e47f4427fcb3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:10.950354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:10.963518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b624e0-6979-4da4-9096-f48f237ae628 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.055628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.056671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.057275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.057619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] PATH_INFO: `/v3/users/783f9a504f7a41be9484d3d2bd42797b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.059420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=783f9a504f7a41be9484d3d2bd42797b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.062920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.124226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783f9a504f7a41be9484d3d2bd42797b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.139793 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783f9a504f7a41be9484d3d2bd42797b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.140626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783f9a504f7a41be9484d3d2bd42797b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.150682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783f9a504f7a41be9484d3d2bd42797b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.159660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ec72f318-bc03-4803-81d8-6331d610eacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783f9a504f7a41be9484d3d2bd42797b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.166279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1091/2160] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:10 2022] DELETE /identity/v3/users/783f9a504f7a41be9484d3d2bd42797b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:11.198992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.201454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.225237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e53b9-f2ed-4fb6-9ee7-a86d3338ab28 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.306400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.307335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.307751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.308186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] PATH_INFO: `/v3/projects/1e987dd9cf634d269c852374bf501a87` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.309439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e987dd9cf634d269c852374bf501a87)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.311583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.367027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e987dd9cf634d269c852374bf501a87', 'request_id': 'req-7f7a106a-6491-4894-ae52-1486c4cc733e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.383391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e987dd9cf634d269c852374bf501a87', 'request_id': 'req-7f7a106a-6491-4894-ae52-1486c4cc733e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.405882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e987dd9cf634d269c852374bf501a87', 'request_id': 'req-7f7a106a-6491-4894-ae52-1486c4cc733e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.425234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] The token cache is being invalidate because project 1e987dd9cf634d269c852374bf501a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:11.425784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.426665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.428819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1070/2161] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:11 2022] DELETE /identity/v3/projects/1e987dd9cf634d269c852374bf501a87 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:11.434399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28787-cb40-48f2-b223-cb7521318b7f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.434679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28787-cb40-48f2-b223-cb7521318b7f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.435101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28787-cb40-48f2-b223-cb7521318b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.459310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-23c28787-cb40-48f2-b223-cb7521318b7f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:11.460553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-23c28787-cb40-48f2-b223-cb7521318b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.492551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:11.534630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1092/2162] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:11.543245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.543930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.544224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.544509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.545443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.559213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a106a-6491-4894-ae52-1486c4cc733e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.565741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.565741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.565741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.565741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] PATH_INFO: `/v3/projects/702402c1ae744f898c1e172358154e41` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.567048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702402c1ae744f898c1e172358154e41)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.569005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.570813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:11.572079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.582919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1093/2163] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:11.588177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.600961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a631d-fb01-47d9-b222-cb923a10196a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.606428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.607188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.607478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.607760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.608530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:11.608870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.610939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.622446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:11.625717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1094/2164] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:43:11.629835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.634824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702402c1ae744f898c1e172358154e41', 'request_id': 'req-22931c55-c5b8-424e-8f26-5801914696d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.641172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab924c6-9f02-4aa0-ab2e-31454703fce7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.643601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702402c1ae744f898c1e172358154e41', 'request_id': 'req-22931c55-c5b8-424e-8f26-5801914696d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.650053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702402c1ae744f898c1e172358154e41', 'request_id': 'req-22931c55-c5b8-424e-8f26-5801914696d5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.663599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] The token cache is being invalidate because project 702402c1ae744f898c1e172358154e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:11.663887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.664546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22931c55-c5b8-424e-8f26-5801914696d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:11.666088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1071/2165] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:11 2022] DELETE /identity/v3/projects/702402c1ae744f898c1e172358154e41 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:11.672389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1121d-7656-4200-91d8-a8d30bacdb40 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.672682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1121d-7656-4200-91d8-a8d30bacdb40 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.673012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1121d-7656-4200-91d8-a8d30bacdb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.692810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f4b1121d-7656-4200-91d8-a8d30bacdb40 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:11.694056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f4b1121d-7656-4200-91d8-a8d30bacdb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.719392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.720230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.720513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.720923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.721647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.723850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.757275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1095/2166] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:43:11.766280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.769072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.775136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1072/2167] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:11.782081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.782385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.782722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.811575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:11.812824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:11.876846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f7d9-674a-494f-8c53-d05a881a311e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.884189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.885839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.886472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.887648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.888843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.891471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.907905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1073/2168] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:11.913351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.924907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a90f1-cbef-4241-9a6c-d8dcaabfef01 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.926918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1096/2169] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:11.928012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.930503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.931424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.932498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.932498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.932832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:11.933347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.935096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.941328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07933-6175-4423-bf71-528e07b375e8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.948368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.949414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1074/2170] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:43:11.950299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.950778 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.951286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.952470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.953975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.955290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.967575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349230c0-c70e-4d12-a072-fc9eb9b5e812 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.973856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:11.974780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:11.975160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:11.975514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:11.975631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1097/2171] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:11.976548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:11.976687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:11.978806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:11.991187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a76b1-7d8d-4104-98b3-df89d1af8567 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:11.999370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.000200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.000604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.001006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.001649 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.003422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.013196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1075/2172] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:11 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:12.014222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.017059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1098/2173] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:12.020892 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.029229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:12.034997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b60e75-9045-40e7-a25d-7ddb1b303d81 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.061963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:12.106093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9f80-43b1-4e5c-810b-b8c981dd1346 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.130095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.130095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.130095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.130095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] PATH_INFO: `/v3/projects/2b043de82f1b43f78e3b92d375111d91/users/7f8df9faa4dd4eeeb724e891cdb55616/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.131382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.132082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.132364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.132637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.133335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.135305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.159936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b043de82f1b43f78e3b92d375111d91, user_id=7f8df9faa4dd4eeeb724e891cdb55616, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.163115 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424403fc-ff3f-44fb-bcd5-f6b32e6511c8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.173596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1076/2174] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:12 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:12.178000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.180437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1099/2175] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:12 2022] PUT /identity/v3/projects/2b043de82f1b43f78e3b92d375111d91/users/7f8df9faa4dd4eeeb724e891cdb55616/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:12.186097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.187160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.187447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.193752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ba97-9355-4067-9dc3-48f4052ba5f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.208973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:12.210277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:12.305438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.306239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.306590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.306943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.307865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.309769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.320278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1100/2176] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:12.325074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1077/2177] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:43:12.330381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.333935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.335880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:12.353730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53ea36-bf6e-47b3-82f3-c4d05d793bdd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.360385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.361175 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.361466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.361884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.363029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.365683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.378560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1101/2178] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:12.383222 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.394241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56e90-a7e2-4b2d-a201-0469fffe0bad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.399785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.402615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.402920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.403238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] PATH_INFO: `/v3/projects/26b39aee6f734447809c87d967410487/users/3b961f2bec7d4b23a5ead9ff65d3b08f/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.411942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295c03-29db-4830-824d-a0d49b512757 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.415762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.417286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.417286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.417286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.426724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.428220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7504d46-a79d-482e-a8ce-4d0ccc9746d9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.431243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b39aee6f734447809c87d967410487, user_id=3b961f2bec7d4b23a5ead9ff65d3b08f, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.435992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.438787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1078/2179] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:43:12.468191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1102/2180] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:12 2022] PUT /identity/v3/projects/26b39aee6f734447809c87d967410487/users/3b961f2bec7d4b23a5ead9ff65d3b08f/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:12.483534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa8c5-5ace-4c73-ab0d-ac89b5741b97 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.483832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa8c5-5ace-4c73-ab0d-ac89b5741b97 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.484117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748aa8c5-5ace-4c73-ab0d-ac89b5741b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.542170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-748aa8c5-5ace-4c73-ab0d-ac89b5741b97 None None] MFA Rules not processed for user `3b961f2bec7d4b23a5ead9ff65d3b08f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:12.552902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-748aa8c5-5ace-4c73-ab0d-ac89b5741b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:12.653450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:12.697161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1079/2181] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:12.707190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:12.710049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:12.821938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffd44-f48c-41b8-b6ba-1fae7f094b55 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:12.826772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:12.827769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:12.828069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:12.828340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:12.836099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:12.840114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:12.842574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:12.856960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1103/2182] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:13.281316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1862427-4570-439f-9261-685508facc55 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:13.281869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1862427-4570-439f-9261-685508facc55 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:13.281950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1862427-4570-439f-9261-685508facc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:13.366004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e1862427-4570-439f-9261-685508facc55 None None] MFA Rules not processed for user `7f8df9faa4dd4eeeb724e891cdb55616`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:13.377617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e1862427-4570-439f-9261-685508facc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:13.482660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1080/2183] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:13.493263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:13.495021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:13.514217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a94dd-8548-453d-a9c2-c4e69f5bea9a service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:13.522142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:13.523751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:13.524107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:13.524448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:13.539369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:13.551199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87670b7-81fd-4efc-bcc1-260905fa252b service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:13.568822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1104/2184] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:43:14.419859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:14.419859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:14.419859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:14.437949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] MFA Rules not processed for user `1e4d2ec6d0014c99a818e1000869b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:14.439176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:14.496545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1081/2185] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:14.992732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82602a7-65d0-4dce-98ae-79143075e5e0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:14.993226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82602a7-65d0-4dce-98ae-79143075e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:14.993340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82602a7-65d0-4dce-98ae-79143075e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:15.024237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c82602a7-65d0-4dce-98ae-79143075e5e0 None None] MFA Rules not processed for user `7f8df9faa4dd4eeeb724e891cdb55616`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:15.025675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c82602a7-65d0-4dce-98ae-79143075e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:15.035787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1105/2186] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:19.808619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:19.810361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:19.909773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa28c1b-5923-4c44-8382-158d374ea9c5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:19.916005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:19.917230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:19.917755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:19.918308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] PATH_INFO: `/v3/users/d50acc7f684a47b78771ad57ab9d91b7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:19.919760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50acc7f684a47b78771ad57ab9d91b7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:19.922829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:19.969448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:20.006617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50acc7f684a47b78771ad57ab9d91b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.016219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50acc7f684a47b78771ad57ab9d91b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.026689 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50acc7f684a47b78771ad57ab9d91b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.027637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50acc7f684a47b78771ad57ab9d91b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.028310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50acc7f684a47b78771ad57ab9d91b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.049449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1082/2187] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:19 2022] DELETE /identity/v3/users/d50acc7f684a47b78771ad57ab9d91b7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:20.054054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.054519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.054635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.073343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:20.074720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:20.137325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1106/2188] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:20.147121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:20.148802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:20.211109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60045a6b-9818-480f-8d33-2274fcd5ecab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:20.215525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:20.216194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.216539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.216754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.233765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:20.234974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455028e7-a37a-4fe3-9c7a-7232cdfde5b9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:20.246457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1083/2189] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:20.536803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:20.538426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:20.561116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e91ffe-0320-4f01-b5c2-40df4c783108 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:20.567409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:20.568367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.568651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.568928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] PATH_INFO: `/v3/projects/b9016db2909b48ebb0e60cd353c4feeb` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.570150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9016db2909b48ebb0e60cd353c4feeb)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:20.572170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:20.620086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9016db2909b48ebb0e60cd353c4feeb', 'request_id': 'req-1c126d2c-6d85-467b-9d70-65203abb6337'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.627399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9016db2909b48ebb0e60cd353c4feeb', 'request_id': 'req-1c126d2c-6d85-467b-9d70-65203abb6337'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.636289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9016db2909b48ebb0e60cd353c4feeb', 'request_id': 'req-1c126d2c-6d85-467b-9d70-65203abb6337'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.647137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] The token cache is being invalidate because project b9016db2909b48ebb0e60cd353c4feeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:20.647428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.648083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c126d2c-6d85-467b-9d70-65203abb6337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:20.649677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1107/2190] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:20 2022] DELETE /identity/v3/projects/b9016db2909b48ebb0e60cd353c4feeb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:20.654765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.655067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.655394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.674201 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:20.675450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:20.753184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1084/2191] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:20.761592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.761858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.762157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.782674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:20.783888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:20.793449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1108/2192] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:20.798583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:20.814988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4c2c-2549-4bfe-aaa9-ca6c8d02f25e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:20.823394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:20.824256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.824424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.824732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.825501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:20.825807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:20.827448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:20.840708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1085/2193] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:43:20.845335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:20.855801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50708-da7e-4b3b-855e-451d640b4748 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:20.861697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:20.862365 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.862652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.862932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.863668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:20.865604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:20.902753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1109/2194] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:20 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:20.908697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:20.923928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4dad6-b8ed-42a1-90ee-87126ebbe488 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:20.959508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:20.997370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:20.998119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:20.998355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:20.998728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:20.999349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.001320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.025809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1086/2195] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:20 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:21.030702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.041713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b531d-5a12-4ed7-b557-44a31d641e55 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.049763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.050701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.051606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.051959 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.053067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.057114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.075815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1110/2196] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:21.080910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.098117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e18d0b-da75-4001-9e55-fca9c9444b31 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.104297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.105064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.105655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.105930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.106686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.108543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5c7d1-c290-40bb-b463-fccdc8768ce8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.120150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1087/2197] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:43:21.125514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.137796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de8f81-8cfb-466c-b90f-f03b5b2cd2a7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.145490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.146399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.146681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.146955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] PATH_INFO: `/v3/projects/6e40b15b7fee4e38b86da6e7fdf09b7f/users/4de179d794f947178654637c53e254f0/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.171210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e40b15b7fee4e38b86da6e7fdf09b7f, user_id=4de179d794f947178654637c53e254f0, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.175687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.184373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:21.198287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1111/2198] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:21 2022] PUT /identity/v3/projects/6e40b15b7fee4e38b86da6e7fdf09b7f/users/4de179d794f947178654637c53e254f0/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:21.209631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.209932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.210242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.241537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] MFA Rules not processed for user `4de179d794f947178654637c53e254f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:21.242768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:21.321261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1088/2199] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:21.326075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.397443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7621e-c816-4177-b282-3df42255afb6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.409476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.410181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.410466 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.410741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.411504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.413651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.447304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1112/2200] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:21 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:21.453188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.466554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745aff8-9626-4f1d-a216-1afe56cc94e1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.548606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.549325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.549609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.550056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.551226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.554018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.584552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1089/2201] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:21 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:21.586126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.600560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e8130-fb4d-4c34-8542-7fc59313291f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.609393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.610786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.611152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.611452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.612196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.614645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.629886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1113/2202] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:21.634599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.645600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76610ec-ca37-4b4c-9269-d9a966e78860 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.651929 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.652850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.653138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.653416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] PATH_INFO: `/v3/projects/ff3292876b9e44cc9763de12da8f551e/users/4c0083d367534f989486467dfa52253d/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.671356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3292876b9e44cc9763de12da8f551e, user_id=4c0083d367534f989486467dfa52253d, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.674154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.695775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1090/2203] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:21 2022] PUT /identity/v3/projects/ff3292876b9e44cc9763de12da8f551e/users/4c0083d367534f989486467dfa52253d/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:21.699245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.715016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fdc88-5792-439d-9678-5dcb0c377dce admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.787449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.788136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.788417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.788688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.789416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.791034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5e71c-ddf3-443c-a453-94eb44cfb717 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.801936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1114/2204] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:21.806254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.817287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1ee96-2eff-49db-bce4-c810a3cff9a2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:21.821799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:21.822685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.822967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.823283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] PATH_INFO: `/v3/projects/ff3292876b9e44cc9763de12da8f551e/users/4c0083d367534f989486467dfa52253d/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.825825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3292876b9e44cc9763de12da8f551e, user_id=4c0083d367534f989486467dfa52253d, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:21.828516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:21.830761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:21.851576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1091/2205] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:21 2022] PUT /identity/v3/projects/ff3292876b9e44cc9763de12da8f551e/users/4c0083d367534f989486467dfa52253d/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:21.863992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:21.864331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:21.864626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:21.891431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] MFA Rules not processed for user `4c0083d367534f989486467dfa52253d`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:21.892637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:21.944397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:21.974802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1115/2206] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:21.987167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:21.988987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:22.049483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff64be-ef41-4d3b-87b5-37824013eac9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:22.053326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:22.053981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:22.054265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:22.054543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:22.066524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:22.068108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:22.080995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1092/2207] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:22.155464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:22.165807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c6e0a-0129-4eb8-a7e6-2ca5ebead67f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:22.170152 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:22.170848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:22.171173 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:22.171454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:22.233001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:22.234579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f8460-0f60-4bfd-a547-4864454d05c0 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:22.244767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1116/2208] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:31.198289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:31.267523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed4ad-9f63-4cda-8d3e-90dc516afec1 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:31.272482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:31.273407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:31.273690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:31.273965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] PATH_INFO: `/v3/users/4de179d794f947178654637c53e254f0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:31.275221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de179d794f947178654637c53e254f0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:31.277204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:31.328021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de179d794f947178654637c53e254f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:31.335590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de179d794f947178654637c53e254f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:31.344463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de179d794f947178654637c53e254f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:31.345668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de179d794f947178654637c53e254f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:31.346522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de179d794f947178654637c53e254f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:31.368192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1093/2209] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:31 2022] DELETE /identity/v3/users/4de179d794f947178654637c53e254f0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:31.372871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382dd05-f891-4187-9a60-171895400adc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:31.373437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382dd05-f891-4187-9a60-171895400adc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:31.373437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382dd05-f891-4187-9a60-171895400adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:31.391738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c382dd05-f891-4187-9a60-171895400adc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:31.393109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c382dd05-f891-4187-9a60-171895400adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:31.461357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1117/2210] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:31.471240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:31.472699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:31.493909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:31.553223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc2f17-980b-4b30-90aa-7e9476126824 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:31.557865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:31.558812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:31.559309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:31.560210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:31.573033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:31.574648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:31.590543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1094/2211] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:31.944924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382dd05-f891-4187-9a60-171895400adc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:31.946519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c382dd05-f891-4187-9a60-171895400adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:31.958924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382dd05-f891-4187-9a60-171895400adc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:31.963800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:31.964690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:31.964973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:31.965403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] PATH_INFO: `/v3/projects/6e40b15b7fee4e38b86da6e7fdf09b7f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:31.966629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e40b15b7fee4e38b86da6e7fdf09b7f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:31.968592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:32.021293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e40b15b7fee4e38b86da6e7fdf09b7f', 'request_id': 'req-daef3252-a0ac-4de3-bf53-b0257d276687'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.027449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e40b15b7fee4e38b86da6e7fdf09b7f', 'request_id': 'req-daef3252-a0ac-4de3-bf53-b0257d276687'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.036721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e40b15b7fee4e38b86da6e7fdf09b7f', 'request_id': 'req-daef3252-a0ac-4de3-bf53-b0257d276687'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.047351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] The token cache is being invalidate because project 6e40b15b7fee4e38b86da6e7fdf09b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:32.047636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.048315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.050029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1118/2212] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:31 2022] DELETE /identity/v3/projects/6e40b15b7fee4e38b86da6e7fdf09b7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:32.058063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:32.068665 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cc99b-7289-4db3-ac4d-661af72e1093 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:32.132145 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:32.133159 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.133446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.133727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] PATH_INFO: `/v3/users/4c0083d367534f989486467dfa52253d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.134934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0083d367534f989486467dfa52253d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:32.136937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:32.201393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0083d367534f989486467dfa52253d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.209459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0083d367534f989486467dfa52253d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.217508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0083d367534f989486467dfa52253d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.218211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0083d367534f989486467dfa52253d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.218903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9f979256-3bea-46e3-aa43-e0669164e2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0083d367534f989486467dfa52253d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.238997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1095/2213] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:32 2022] DELETE /identity/v3/users/4c0083d367534f989486467dfa52253d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:32.521453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:32.522947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:32.538611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef3252-a0ac-4de3-bf53-b0257d276687 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:32.549465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:32.602484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:32.603288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.603559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.603835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] PATH_INFO: `/v3/projects/ff3292876b9e44cc9763de12da8f551e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.605401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3292876b9e44cc9763de12da8f551e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:32.607347 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:32.669854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3292876b9e44cc9763de12da8f551e', 'request_id': 'req-c075f7cd-8609-45b2-b2d6-f94d17e474f2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.679041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3292876b9e44cc9763de12da8f551e', 'request_id': 'req-c075f7cd-8609-45b2-b2d6-f94d17e474f2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.688053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3292876b9e44cc9763de12da8f551e', 'request_id': 'req-c075f7cd-8609-45b2-b2d6-f94d17e474f2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.699166 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] The token cache is being invalidate because project ff3292876b9e44cc9763de12da8f551e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:32.699478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.700128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c075f7cd-8609-45b2-b2d6-f94d17e474f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:32.701757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1119/2214] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:32 2022] DELETE /identity/v3/projects/ff3292876b9e44cc9763de12da8f551e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:32.706892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.707233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.707515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.758189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:32.759402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:32.819614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:32.864039 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1096/2215] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:32.872133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.872133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.872529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.895790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:32.897150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:32.905453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1120/2216] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:32.910332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:32.920162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df085cbb-fe7e-472c-b6ee-5fe059caf19f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:32.925876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:32.926809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.927206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.927559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.928377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:32.928756 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:32.930467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:32.947125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1097/2217] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:43:32.948918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:32.958987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1e1d5-fe48-4f10-9147-588faa602fbf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:32.963912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:32.964587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:32.964875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:32.965164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:32.965974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:32.967941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:32.999958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1121/2218] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:32 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:33.001216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.014500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3e96d-01ca-45b0-8fcf-df6e7ceb0e04 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.081498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.082195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.082485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.082767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.083496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.085486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.129655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1098/2219] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:33 2022] POST /identity/v3/users => generated 279 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:33.131022 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.142469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f912-a47d-4fdc-8595-785cbaa638a3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.147345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.148031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.148317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.148593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.149353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.150984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.161883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1122/2220] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:33.167920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.183591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a125ae7-1a03-4de3-aa1f-b5bca280f794 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.191009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.191705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.191988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.192263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.192993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.194592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc9096-bfa2-42ad-957b-115689397704 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.204562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1099/2221] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:33.209250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.219352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05c5ec-8329-45f3-9e73-ed2bc0fb0e0e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.225097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.226460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.226842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.227283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] PATH_INFO: `/v3/projects/61adb98e516e407f9afdab4a9f39ecaf/users/782f2950eaf94dc2a8c9a74a8a9bde03/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.246929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=61adb98e516e407f9afdab4a9f39ecaf, user_id=782f2950eaf94dc2a8c9a74a8a9bde03, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.249641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.265021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1123/2222] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:33 2022] PUT /identity/v3/projects/61adb98e516e407f9afdab4a9f39ecaf/users/782f2950eaf94dc2a8c9a74a8a9bde03/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:33.275912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.276205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.276484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.313379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] MFA Rules not processed for user `782f2950eaf94dc2a8c9a74a8a9bde03`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:33.315651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:33.411024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1100/2223] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:33 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:33.412171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.458437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:33.483700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8d4d3-259d-4025-aae2-331cf8eced8d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.490300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.491112 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.491704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.492037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.492812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.495099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.533096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1124/2224] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:33 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:43:33.539251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.556283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a9c54-b46d-4818-bf42-3550a317b3ee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.605462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:33.639679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.640304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.640587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.640862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.641562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.643628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.681930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1101/2225] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:33 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:33.687293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.700857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b00fc9-df08-4417-94dc-a836c8b565d8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.706342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.707031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.707346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.707659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.708427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.710151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.725729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1125/2226] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:33.726909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.739470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5bc83-3908-4b79-a502-a527b07cd04b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.745503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.746651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.746939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.747252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] PATH_INFO: `/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776/users/fa6e77df21cb44a8bd02f39975d976e8/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.765579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4a9cc35a1e47b5a8848cea1b5a7776, user_id=fa6e77df21cb44a8bd02f39975d976e8, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.769684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.788901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1102/2227] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:33 2022] PUT /identity/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776/users/fa6e77df21cb44a8bd02f39975d976e8/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:43:33.791514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.805311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c6016-392f-4dbf-9587-32393c4040da admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.872295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.873019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.873300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.873579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.874311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.875967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6d938-c16b-4bcf-abc0-7b78ebe2a9fc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.891043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1126/2228] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:33.895106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:33.905824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e932-c1d1-445a-8953-cd4d4ac7f911 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:33.910916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:33.911844 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.912134 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.912407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] PATH_INFO: `/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776/users/fa6e77df21cb44a8bd02f39975d976e8/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.914356 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4a9cc35a1e47b5a8848cea1b5a7776, user_id=fa6e77df21cb44a8bd02f39975d976e8, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:33.917060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:33.932895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1103/2229] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:33 2022] PUT /identity/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776/users/fa6e77df21cb44a8bd02f39975d976e8/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:33.947823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:33.948206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:33.948511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:33.981252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] MFA Rules not processed for user `fa6e77df21cb44a8bd02f39975d976e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:33.983011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:34.063717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1127/2230] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:33 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:34.075715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:34.077666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:34.148586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23be6b6-2671-4491-a70b-3ddc6d5a5088 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:34.156583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:34.157492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:34.158853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:34.159231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:34.175153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:34.176850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07591840-5ab0-412e-913b-297e106ab626 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:34.188474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1104/2231] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:34 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:34.266916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:34.276757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396f6d2-0989-4265-b0ed-3d3033499c75 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:34.280332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:34.281065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:34.281353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:34.281645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:34.343547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:34.398379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:34.398379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c23e86-b797-4fde-b980-588413f1b8f4 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:34.405145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1128/2232] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:34 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:39.638446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:39.639010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:39.639010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:39.658016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] MFA Rules not processed for user `d5e6d9fad06449c1b43254df442d224f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:39.659282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:39.720772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1105/2233] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:44.400052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=12 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:49.712780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:51.150930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:51.150930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:51.150930 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:51.165446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:51.166660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:51.245343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1129/2234] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:43:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:51.294017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:51.295653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:51.391598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b631703-3c62-4d74-8cbe-664ea52f1858 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:51.397658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:51.398616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:51.398919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:51.399237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:51.409596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:51.411835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97863df-0794-4061-8bc1-50d29d09d867 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:51.421761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1106/2235] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:43:52.520679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:52.590181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87801-db1c-4666-951b-5b0d7ee061b8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:52.595587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:52.596533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:52.596885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:52.597169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] PATH_INFO: `/v3/users/782f2950eaf94dc2a8c9a74a8a9bde03` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:52.598413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] RBAC: Authorizing `identity:delete_user(user_id=782f2950eaf94dc2a8c9a74a8a9bde03)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:52.600382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:52.653058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782f2950eaf94dc2a8c9a74a8a9bde03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:52.667053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782f2950eaf94dc2a8c9a74a8a9bde03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:52.668165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782f2950eaf94dc2a8c9a74a8a9bde03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:52.678716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782f2950eaf94dc2a8c9a74a8a9bde03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:52.690072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782f2950eaf94dc2a8c9a74a8a9bde03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:52.700733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1130/2236] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:52 2022] DELETE /identity/v3/users/782f2950eaf94dc2a8c9a74a8a9bde03 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:52.705432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:52.706728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:52.707192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:52.728846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:52.730741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:52.809307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1107/2237] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:52.820729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:52.822345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:52.856029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:52.892136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad0c5d-8deb-4c3d-9fb8-7352b4db542f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:52.897481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:52.898325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:52.899011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:52.900218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:52.916557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:52.918077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:52.931885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1131/2238] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:43:53.269546 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:53.271492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:53.284415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef85e-dd6e-4fd0-8543-7960ae0f9a89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:53.290727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:53.291616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:53.291911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:53.292285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] PATH_INFO: `/v3/projects/61adb98e516e407f9afdab4a9f39ecaf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:53.293933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61adb98e516e407f9afdab4a9f39ecaf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:53.296066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:53.364034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61adb98e516e407f9afdab4a9f39ecaf', 'request_id': 'req-a4e028c1-d42a-423a-887d-83357fb9d625'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.379706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61adb98e516e407f9afdab4a9f39ecaf', 'request_id': 'req-a4e028c1-d42a-423a-887d-83357fb9d625'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.390961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61adb98e516e407f9afdab4a9f39ecaf', 'request_id': 'req-a4e028c1-d42a-423a-887d-83357fb9d625'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.412268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] The token cache is being invalidate because project 61adb98e516e407f9afdab4a9f39ecaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:53.412969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.413840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.415901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1108/2239] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:53 2022] DELETE /identity/v3/projects/61adb98e516e407f9afdab4a9f39ecaf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:53.420833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:53.435206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e96907-48fc-4de9-b5fc-cc232e3c72fc service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:53.546740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:53.547388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:53.547388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:53.547592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] PATH_INFO: `/v3/users/fa6e77df21cb44a8bd02f39975d976e8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:53.550792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6e77df21cb44a8bd02f39975d976e8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:53.552818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:53.629588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6e77df21cb44a8bd02f39975d976e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.644209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6e77df21cb44a8bd02f39975d976e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.645260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6e77df21cb44a8bd02f39975d976e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.654824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6e77df21cb44a8bd02f39975d976e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.670472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4f9995df-8265-41b1-9be0-363b5d77f665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6e77df21cb44a8bd02f39975d976e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:53.680143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1132/2240] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:43:53 2022] DELETE /identity/v3/users/fa6e77df21cb44a8bd02f39975d976e8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:54.019326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.021552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:54.037296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e028c1-d42a-423a-887d-83357fb9d625 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.057682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:54.115525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.116655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.117085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.117499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] PATH_INFO: `/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.119527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4a9cc35a1e47b5a8848cea1b5a7776)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.121663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.172873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4a9cc35a1e47b5a8848cea1b5a7776', 'request_id': 'req-6122151e-c644-4637-af6e-31d648d96ed4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:54.186877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4a9cc35a1e47b5a8848cea1b5a7776', 'request_id': 'req-6122151e-c644-4637-af6e-31d648d96ed4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:54.195484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4a9cc35a1e47b5a8848cea1b5a7776', 'request_id': 'req-6122151e-c644-4637-af6e-31d648d96ed4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:54.208750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] The token cache is being invalidate because project 0d4a9cc35a1e47b5a8848cea1b5a7776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:43:54.208969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:54.209841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6122151e-c644-4637-af6e-31d648d96ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:43:54.211909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1109/2241] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:43:54 2022] DELETE /identity/v3/projects/0d4a9cc35a1e47b5a8848cea1b5a7776 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:54.217295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.217585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.217925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.239552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:54.241107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:54.300567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:54.326720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1133/2242] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:54.335422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.335819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.336232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.362307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:54.363987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:54.374875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1110/2243] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:54.382450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.393541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f32e49-5b3c-4a53-8df5-6aa11ee2a34d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.398901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.399756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.400140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.400666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.401629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:43:54.402078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.403956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.420438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1134/2244] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:43:54.421568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.441770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84edd8f-7f2e-48f5-a551-dded270a5532 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.447108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.447793 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.447931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.448176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.449198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.451152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.497725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1111/2245] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:54 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:54.502195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.515155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e11046-e1fc-4885-bf8d-1aae82b3fe80 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.593339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.594207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.594566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.595013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.596079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.598131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.638319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1135/2246] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:54 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:54.641431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.665872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab683c-9c40-4b83-ac5c-652f812b897c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.675066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.675937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.676935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.677291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.678338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.680425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.708491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1112/2247] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:54.713122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.729518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7b47-8b5b-4b92-a9b6-c5bca4ceaded admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.739951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.740710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.745553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.745553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.745553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.745553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d4ab8-4839-4de6-9694-67566e7f5ba1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.798659 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1136/2248] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:54.802980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:54.837746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa5c3-4862-4fb9-95bd-0eaabd9fd268 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:54.843051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:54.844087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.844489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.844782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] PATH_INFO: `/v3/projects/9e41da7a67a544e5a7149952ce9ab8eb/users/43da164734c14b73ae29f0f3ec4c3f07/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.863110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e41da7a67a544e5a7149952ce9ab8eb, user_id=43da164734c14b73ae29f0f3ec4c3f07, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:54.866424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:54.892997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1113/2249] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:54 2022] PUT /identity/v3/projects/9e41da7a67a544e5a7149952ce9ab8eb/users/43da164734c14b73ae29f0f3ec4c3f07/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:54.906297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:54.906679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:54.907037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:54.936223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] MFA Rules not processed for user `43da164734c14b73ae29f0f3ec4c3f07`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:54.937651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:55.005767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1137/2250] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:55.010157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.054829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:55.080399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e789e-8e46-4bc8-8528-08ec68011edf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.085852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.086545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.086876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.087169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.087900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.089870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.126481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1114/2251] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:43:55 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:55.127867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.138789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d037c-543a-4b3e-8949-cc5e7c738d9e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.181182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:43:55.200454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.201194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.201565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.203808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.203808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.204979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.228704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1138/2252] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:43:55 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:43:55.233069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.244192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c709-598f-4d38-bf2c-270621fa4f89 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.249100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.249770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.250057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.250339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.251100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.252757 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.264017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1115/2253] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:55.268622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.279747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41f4d-982b-4d82-902d-5f85dd37904d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.285079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.286015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.286300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.286581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] PATH_INFO: `/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118/users/266e3fc4b4dc43b2aeecba3ea50e1943/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.303787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec0234aa2e340d3bcac8bbc156f7118, user_id=266e3fc4b4dc43b2aeecba3ea50e1943, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.306484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.323028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1139/2254] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:55 2022] PUT /identity/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118/users/266e3fc4b4dc43b2aeecba3ea50e1943/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:55.327094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.337654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668d59-2bf6-403d-a91b-77e956515197 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.453978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.456207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.456530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.456837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.457681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.459395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b524-fc51-4cb8-82c7-9d345c0eed50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.499475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1116/2255] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:43:55.504379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.528756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb33f-82b8-4f08-b960-811c84d62cb7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.537501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.539494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.539893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.540259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] PATH_INFO: `/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118/users/266e3fc4b4dc43b2aeecba3ea50e1943/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.542411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec0234aa2e340d3bcac8bbc156f7118, user_id=266e3fc4b4dc43b2aeecba3ea50e1943, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.545364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.560038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1140/2256] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:43:55 2022] PUT /identity/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118/users/266e3fc4b4dc43b2aeecba3ea50e1943/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:43:55.571952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.572398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.572745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.605470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] MFA Rules not processed for user `266e3fc4b4dc43b2aeecba3ea50e1943`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:43:55.606772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:55.682240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1117/2257] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:43:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:43:55.694283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:43:55.696826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:43:55.774341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc339bbd-8423-4ab8-b99d-992d685760bf admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:43:55.778864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:43:55.779555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:43:55.780269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:43:55.780550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:43:55.848554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:43:55.850570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:43:55.878747 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1141/2258] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:43:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:01.718328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:01.732574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff934fa-87a8-4657-81ee-523fde78a4a3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:01.736572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:01.738037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:01.738331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:01.738612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:01.769454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:01.769454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c544d-bb9d-4924-9a57-fe62fda0383d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:01.780308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1118/2259] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:05.857995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:07.138710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:07.140310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:07.254742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf93473-22dd-4354-a33f-691436675f57 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:07.260486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:07.261724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:07.262007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:07.262280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] PATH_INFO: `/v3/users/3b961f2bec7d4b23a5ead9ff65d3b08f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:07.263615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b961f2bec7d4b23a5ead9ff65d3b08f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:07.265774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:07.390585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b961f2bec7d4b23a5ead9ff65d3b08f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:07.408630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b961f2bec7d4b23a5ead9ff65d3b08f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:07.410068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b961f2bec7d4b23a5ead9ff65d3b08f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:07.420901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b961f2bec7d4b23a5ead9ff65d3b08f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:07.433858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b961f2bec7d4b23a5ead9ff65d3b08f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:07.448121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:07.449409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:07.450026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:07.451736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1142/2260] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:07 2022] DELETE /identity/v3/users/3b961f2bec7d4b23a5ead9ff65d3b08f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:07.471937 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:07.473540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:07.494502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:07.567763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1119/2261] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:07.575478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:07.577777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:07.723121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1a4e8-71ba-4632-b5b5-8cf9bc19d88b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:07.732054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:07.733400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:07.748308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:07.748743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:07.769339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:07.773319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a756e8-1b3d-4647-a2f8-2a8f2bcec724 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:07.797310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1143/2262] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:08.493203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:08.496642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:08.516539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50312968-62d9-4fcd-874b-b9b94877d168 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:08.522477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:08.523896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:08.524360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:08.525309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] PATH_INFO: `/v3/projects/26b39aee6f734447809c87d967410487` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:08.526754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b39aee6f734447809c87d967410487)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:08.530734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:08.626099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b39aee6f734447809c87d967410487', 'request_id': 'req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:08.660641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b39aee6f734447809c87d967410487', 'request_id': 'req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:08.674537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b39aee6f734447809c87d967410487', 'request_id': 'req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:08.698150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] The token cache is being invalidate because project 26b39aee6f734447809c87d967410487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:08.699051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:08.700351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ad5dd5cb-9d2d-4d68-8d11-edf92b743ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:08.707730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:08.708765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:08.709388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:08.710991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1120/2263] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:08 2022] DELETE /identity/v3/projects/26b39aee6f734447809c87d967410487 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:08.733426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:08.735135 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:08.756567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:08.840472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1144/2264] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:44:08.849918 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:08.850281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:08.850631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:08.885337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:08.889324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:08.913122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1121/2265] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:08.918075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:08.935693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903972-c174-4535-8854-6b9ff232f6bf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:08.945615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:08.947703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:08.948184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:08.948849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:08.950762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:08.951438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:08.958532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.024215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1145/2266] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:44:09.029327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.050610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c520de1-9dc2-4a10-a8c4-65e1eeb81064 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.061631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.062319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.062602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.062875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.063606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.071655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.163069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1122/2267] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:09 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:09.168519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.214814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4757-4362-4cd5-8476-ddc637d3de10 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.336101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.337439 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.337902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.338493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.341448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.350342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.424670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1146/2268] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:09 2022] POST /identity/v3/users => generated 295 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:09.426222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.446746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d086f6-b692-45ca-a3fc-a74e605e5dab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.468314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.470676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.472036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.472669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.474287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.476164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.500148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1123/2269] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:09.504576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.516704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86284-efeb-4ae8-802c-d436cfb3f898 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.525627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.527281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.530162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.530596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.531583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.535674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717895a-2303-44c9-9f9d-cf0b17f913d2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.549679 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1147/2270] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:09.553559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.574177 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6462751-2ac3-46cc-a53c-ade243fe45fb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.584517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.586556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.587298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.587810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] PATH_INFO: `/v3/projects/46aacaf31e9d4d80bda1823bb07ecee1/users/e17d40ba9d5046839b7c030d3bb43af6/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.628042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aacaf31e9d4d80bda1823bb07ecee1, user_id=e17d40ba9d5046839b7c030d3bb43af6, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.632222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.658935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1124/2271] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:09 2022] PUT /identity/v3/projects/46aacaf31e9d4d80bda1823bb07ecee1/users/e17d40ba9d5046839b7c030d3bb43af6/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:09.671462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a40bd4-38f0-4c80-9c13-9819f7f2a404 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.671749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a40bd4-38f0-4c80-9c13-9819f7f2a404 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.672031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a40bd4-38f0-4c80-9c13-9819f7f2a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.708947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d0a40bd4-38f0-4c80-9c13-9819f7f2a404 None None] MFA Rules not processed for user `e17d40ba9d5046839b7c030d3bb43af6`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:09.710188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d0a40bd4-38f0-4c80-9c13-9819f7f2a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:09.795308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1148/2272] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:09.806989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:09.810189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:09.912632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:09.933111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a47534-91a3-4eb4-9927-c9abfaab92df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:09.939476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:09.939476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:09.939476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:09.939476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:09.953682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:09.955422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cf596-7970-4642-bd3c-211614919834 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:09.967920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1125/2273] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:11.072404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:11.073361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:11.073789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:11.097425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] MFA Rules not processed for user `fd136fcda9a04a16b258301a86bde8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:11.099247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:11.115487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:11.165451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1149/2274] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:19.962770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:21.158190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=17 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:25.787766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:25.788655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:25.788821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:25.824590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:25.825836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:25.906110 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1126/2275] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:44:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:25.919035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:25.920660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:26.031581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4ea5f-eaaf-4c67-909b-75f57995c4fc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:26.037362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:26.038026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:26.038303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:26.038580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:26.055332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:26.058190 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:26.074895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1150/2276] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:44:33.517379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:33.520002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:33.714953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2218-25f9-4d31-b3d0-714e6d73e87d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:33.722830 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:33.724329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:33.724818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:33.725313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] PATH_INFO: `/v3/users/1e4d2ec6d0014c99a818e1000869b38c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:33.727453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4d2ec6d0014c99a818e1000869b38c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:33.731547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:33.789566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4d2ec6d0014c99a818e1000869b38c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:33.791832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:33.798795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4d2ec6d0014c99a818e1000869b38c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:33.809494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4d2ec6d0014c99a818e1000869b38c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:33.810263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4d2ec6d0014c99a818e1000869b38c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:33.810951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-757106b2-5fe9-436e-9914-05d57e2906d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4d2ec6d0014c99a818e1000869b38c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:33.839936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1127/2277] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:33 2022] DELETE /identity/v3/users/1e4d2ec6d0014c99a818e1000869b38c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:34.111768 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.152418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c464-5715-4b01-adb1-9491210265d3 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.259701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.261296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.261719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.262170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] PATH_INFO: `/v3/projects/6c5b285a04564d5d917a4566bec984b2` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.263658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5b285a04564d5d917a4566bec984b2)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.266311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.328060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5b285a04564d5d917a4566bec984b2', 'request_id': 'req-977669ea-57c4-4551-9512-0d29703f3fc8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:34.334633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5b285a04564d5d917a4566bec984b2', 'request_id': 'req-977669ea-57c4-4551-9512-0d29703f3fc8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:34.338699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:34.344156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5b285a04564d5d917a4566bec984b2', 'request_id': 'req-977669ea-57c4-4551-9512-0d29703f3fc8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:34.363722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] The token cache is being invalidate because project 6c5b285a04564d5d917a4566bec984b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:34.364146 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:34.364990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-977669ea-57c4-4551-9512-0d29703f3fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:34.366760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1151/2278] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:34 2022] DELETE /identity/v3/projects/6c5b285a04564d5d917a4566bec984b2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:34.372451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.372867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.373238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.445139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:34.447340 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:34.521990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1128/2279] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:34.529935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.530411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.530787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.552949 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:34.554192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:34.563263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1152/2280] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:34.568208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.583552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefed46-35a6-4789-929f-6ff24f537244 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.589175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.589915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.590511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.590661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.591423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:34.591771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.593406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.607497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1129/2281] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:44:34.613307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.623182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c7120-4d0f-446f-b148-c8e1f622d8f6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.629001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.629795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.630272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.630640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.631569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.633818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.670458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1153/2282] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:34.674391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.685379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b476fd-2ae6-475d-b3d4-bea1e144e83c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.803628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.804248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.804524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.805188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.806047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.808049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.841820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1130/2283] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:34 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:44:34.848653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.862975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875cff-a965-47aa-9da3-d37763301aa5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.867948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.868817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.869128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.869428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.870230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.873562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.890762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1154/2284] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:34.893555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.905990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfd297-27ff-4d42-ab16-cea7c213b9f6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.911871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.912714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.913165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.913558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.914472 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.916455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6cdfb-1db0-47ad-801a-b6545cd27095 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:34.936688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1131/2285] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:34.938651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:34.961821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3684c-2c20-473f-9580-aea9a433da0b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:34.967751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:34.968270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:34.968813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:34.969746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] PATH_INFO: `/v3/projects/d0396f3f4a90429fa40065d815ac1aa5/users/b4f77464774649d5bab2c94f612ea435/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:34.992479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0396f3f4a90429fa40065d815ac1aa5, user_id=b4f77464774649d5bab2c94f612ea435, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:34.995554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:35.011917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1155/2286] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:34 2022] PUT /identity/v3/projects/d0396f3f4a90429fa40065d815ac1aa5/users/b4f77464774649d5bab2c94f612ea435/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:35.022612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:35.022891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:35.023207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:35.029041 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:35.060594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] MFA Rules not processed for user `b4f77464774649d5bab2c94f612ea435`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:35.062674 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:35.155890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1132/2287] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:44:35.169916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:35.172238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:35.263311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40374e3b-36ac-48cd-aa41-513f250165a8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:35.268338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:35.269528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:35.269814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:35.270092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:35.281561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:35.284488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:35.301367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1156/2288] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:44:35.937998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:35.940570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:35.984561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:35.988334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.034251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e92e05-e31e-42ac-8aa2-777676960b6a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:36.040102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:36.041009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.041322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.041604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] PATH_INFO: `/v3/users/e17d40ba9d5046839b7c030d3bb43af6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.043907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17d40ba9d5046839b7c030d3bb43af6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:36.047677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:36.134463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695d54-0c55-4979-9a0e-5134a5af476c service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:36.140598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:36.141650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.142650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.143314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] PATH_INFO: `/v3/users/43da164734c14b73ae29f0f3ec4c3f07` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.144689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43da164734c14b73ae29f0f3ec4c3f07)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:36.147181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:36.176979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17d40ba9d5046839b7c030d3bb43af6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.193304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17d40ba9d5046839b7c030d3bb43af6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.210122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17d40ba9d5046839b7c030d3bb43af6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.212485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17d40ba9d5046839b7c030d3bb43af6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.214338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-042f9a79-8849-44b3-8793-b632dccc5f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17d40ba9d5046839b7c030d3bb43af6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.226042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:36.272493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43da164734c14b73ae29f0f3ec4c3f07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.274005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1133/2289] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:35 2022] DELETE /identity/v3/users/e17d40ba9d5046839b7c030d3bb43af6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:36.279595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f249e-aa98-4098-ad47-4f86e8f52803 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.281057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f249e-aa98-4098-ad47-4f86e8f52803 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.281708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f249e-aa98-4098-ad47-4f86e8f52803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.312196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43da164734c14b73ae29f0f3ec4c3f07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.313816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43da164734c14b73ae29f0f3ec4c3f07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.327299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-162f249e-aa98-4098-ad47-4f86e8f52803 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:36.330005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-162f249e-aa98-4098-ad47-4f86e8f52803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.338373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:36.359016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43da164734c14b73ae29f0f3ec4c3f07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.375605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0b918986-51da-4d63-b0a5-5bab6f1a84b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43da164734c14b73ae29f0f3ec4c3f07'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:36.387007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1157/2290] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:35 2022] DELETE /identity/v3/users/43da164734c14b73ae29f0f3ec4c3f07 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:36.391938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.393088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.393701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.437866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:36.443744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.483157 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1134/2291] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:36.497837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.498360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.501980 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.542846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:36.544629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.574037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1158/2292] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:36.589388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:36.592196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.629000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1135/2293] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:36.629975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:36.631446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.710835 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839789-95d0-4762-bcb0-a89df8bb8726 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:36.717952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:36.719221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.720421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.720421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.727423 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10890d-dd29-4709-891c-f258bc732261 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:36.735218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:36.738075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.739001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:36.739302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.740031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.741602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:36.755967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1159/2294] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:36.757156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:36.758646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:36.773775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:36.774585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:36.795576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1136/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:36.813752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:36.864217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b782d07-5a38-476f-ac92-9967ca396273 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:36.871804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:36.872624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:36.873203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:36.873684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:36.901578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:36.903516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:36.913933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1160/2296] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:44:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:44:37.330862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.337069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.341066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:37.343454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:37.377503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb47ac-9223-4a61-8ac3-16b0a2026d1d service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:37.377740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cc1f5-2a47-4ebb-9f46-5fa5a61fb666 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:37.386660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:37.387858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.388329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.389005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] PATH_INFO: `/v3/projects/9e41da7a67a544e5a7149952ce9ab8eb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.391897 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e41da7a67a544e5a7149952ce9ab8eb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:37.392896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:37.394662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.395574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.395925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] PATH_INFO: `/v3/projects/46aacaf31e9d4d80bda1823bb07ecee1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.397636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aacaf31e9d4d80bda1823bb07ecee1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:37.401000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:37.409791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:37.480521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aacaf31e9d4d80bda1823bb07ecee1', 'request_id': 'req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.489033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:37.490558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aacaf31e9d4d80bda1823bb07ecee1', 'request_id': 'req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.493085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e41da7a67a544e5a7149952ce9ab8eb', 'request_id': 'req-99956c51-6a8e-478f-b22e-ee59ed0a7df1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.505179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aacaf31e9d4d80bda1823bb07ecee1', 'request_id': 'req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.506719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e41da7a67a544e5a7149952ce9ab8eb', 'request_id': 'req-99956c51-6a8e-478f-b22e-ee59ed0a7df1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.515559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e41da7a67a544e5a7149952ce9ab8eb', 'request_id': 'req-99956c51-6a8e-478f-b22e-ee59ed0a7df1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.523170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] The token cache is being invalidate because project 46aacaf31e9d4d80bda1823bb07ecee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:37.523592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.524455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ffa916d8-43d3-4d6e-bef4-d5b0164ecae2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.526154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1161/2297] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:37 2022] DELETE /identity/v3/projects/46aacaf31e9d4d80bda1823bb07ecee1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:37.532766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df474-a73a-4e5d-b685-a1171da4d0ca None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.533155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df474-a73a-4e5d-b685-a1171da4d0ca None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.533658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df474-a73a-4e5d-b685-a1171da4d0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.540857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] The token cache is being invalidate because project 9e41da7a67a544e5a7149952ce9ab8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:37.541143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.541920 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.546373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1137/2298] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:37 2022] DELETE /identity/v3/projects/9e41da7a67a544e5a7149952ce9ab8eb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:37.547416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.549399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:37.555614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-137df474-a73a-4e5d-b685-a1171da4d0ca None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:37.557160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-137df474-a73a-4e5d-b685-a1171da4d0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:37.639321 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1162/2299] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:37.652077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.652415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.652667 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.655283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956c51-6a8e-478f-b22e-ee59ed0a7df1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:37.660636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:37.662621 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.662907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.663231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] PATH_INFO: `/v3/users/266e3fc4b4dc43b2aeecba3ea50e1943` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.664477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=266e3fc4b4dc43b2aeecba3ea50e1943)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:37.666510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:37.682240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:37.683860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:37.697632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1163/2300] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:37.699504 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.719468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a46bc-994e-47b0-80d9-9be36a038e05 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:37.727054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:37.728572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.729079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.729417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.730573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:37.731421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:37.733217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:37.757073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1164/2301] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:44:37.761766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.780010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cca371-ace5-4b2d-a9ca-596ad419c8de admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:37.785122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266e3fc4b4dc43b2aeecba3ea50e1943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.790859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:37.792081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:37.792967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:37.792967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:37.793906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266e3fc4b4dc43b2aeecba3ea50e1943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.794008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:37.798602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:37.810231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266e3fc4b4dc43b2aeecba3ea50e1943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.811726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266e3fc4b4dc43b2aeecba3ea50e1943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.812893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266e3fc4b4dc43b2aeecba3ea50e1943'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:37.847166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1138/2302] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:37 2022] DELETE /identity/v3/users/266e3fc4b4dc43b2aeecba3ea50e1943 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:37.857302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:37.879327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1165/2303] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:37 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:37.883298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:37.885281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.030346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:38.061737 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ce472-e845-4560-98e4-72e4051e2c1d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.174165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.174691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.174979 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.180869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.182081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.186863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.216047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1139/2304] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:37 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:38.220712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.236656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e743ba9-3716-4715-85d8-c7ffee237bd5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.245962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.247506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.247980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.248493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.249635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.252047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.263251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.265075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.269954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1166/2305] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:38.273983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.283859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c625-0f6d-4dd7-b913-c5df41b4d72b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.289173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.290217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.290503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.290783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] PATH_INFO: `/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.292064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec0234aa2e340d3bcac8bbc156f7118)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.294282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.302665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18248d-5c0a-49c1-9d6f-29ac6917aa9d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.312701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.313445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.313780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.314089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.317780 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.319550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.336956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1167/2306] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:38.338361 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.353871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83749c7-8848-4ece-b2fa-12993d8a0ee4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.372675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec0234aa2e340d3bcac8bbc156f7118', 'request_id': 'req-185e7882-007e-4ef1-ae48-1bee2a454798'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:38.385421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec0234aa2e340d3bcac8bbc156f7118', 'request_id': 'req-185e7882-007e-4ef1-ae48-1bee2a454798'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:38.397277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec0234aa2e340d3bcac8bbc156f7118', 'request_id': 'req-185e7882-007e-4ef1-ae48-1bee2a454798'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:38.419705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] The token cache is being invalidate because project 0ec0234aa2e340d3bcac8bbc156f7118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:38.419908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:38.420584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185e7882-007e-4ef1-ae48-1bee2a454798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:38.422197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1140/2307] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:38 2022] DELETE /identity/v3/projects/0ec0234aa2e340d3bcac8bbc156f7118 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:38.427053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42a04-9525-4955-9e09-84e94c343e16 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.427455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42a04-9525-4955-9e09-84e94c343e16 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.427804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42a04-9525-4955-9e09-84e94c343e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.454075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bea42a04-9525-4955-9e09-84e94c343e16 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:38.456331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bea42a04-9525-4955-9e09-84e94c343e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.462650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.466032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.466032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.466032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] PATH_INFO: `/v3/projects/a0aff51028ec477ebbc12e078174977a/users/1636311c73b9461c9a64db18e8fdf0a3/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.486788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0aff51028ec477ebbc12e078174977a, user_id=1636311c73b9461c9a64db18e8fdf0a3, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.493478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1141/2308] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:44:38.494518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfe8ca-a662-43e2-a881-db5b174a95c7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.495333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.495628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.495919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.516065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1168/2309] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:38 2022] PUT /identity/v3/projects/a0aff51028ec477ebbc12e078174977a/users/1636311c73b9461c9a64db18e8fdf0a3/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:38.520242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.520920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.520920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.533147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:38.534395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.542776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:38.544372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.629577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1169/2310] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:38.638073 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.639624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:38.708369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1142/2311] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:38.709490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.722458 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65a36-7f04-423f-9ccb-cf1c47b2e896 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.729377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837210e-9711-4c04-8dab-488eca0b6ff3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.734322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.735682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.736380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.739818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.740862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:38.741252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.743004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.744892 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:38.750832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.750832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.750832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.750832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.773874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.776142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.779899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1143/2312] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:44:38.784166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.791828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1170/2313] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:44:38.801509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e6d27-338b-4225-8d71-318804569699 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.811113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.811113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.811113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.811113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.811584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.813630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:38.858881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:38.865587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1144/2314] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:38 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:38.866172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:38.875230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e680628-6805-4e94-be13-3236920fee58 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:38.958540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:38.959643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:38.959643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:38.959864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:38.960519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:38.964582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.008576 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1171/2315] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:38 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:39.012502 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.051122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765986e5-9068-4908-8483-7101a418dbde admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.129073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.133695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.134188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.134797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.137392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.139379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.167890 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1145/2316] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:39.169107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.181549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321659e5-70d5-4cd4-9cc6-2665f016481f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.189212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.190420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.190697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.190982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.191811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.193664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d8ccf-dc68-4fac-acd0-2504d2fad2e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.210070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1172/2317] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:44:39.211241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.226509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ab4c-edd0-4be4-b1d3-bbd61f5dc5fa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.237604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.248226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.249659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.250107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] PATH_INFO: `/v3/projects/e289ee76ed784a82bb7cbfa6a0cc471f/users/195d647108764778b5779b7225b413fa/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.280483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e289ee76ed784a82bb7cbfa6a0cc471f, user_id=195d647108764778b5779b7225b413fa, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.284038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.325130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1146/2318] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:39 2022] PUT /identity/v3/projects/e289ee76ed784a82bb7cbfa6a0cc471f/users/195d647108764778b5779b7225b413fa/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:39.335566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.336150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.336590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.382308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] MFA Rules not processed for user `195d647108764778b5779b7225b413fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:39.386979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:39.482448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1173/2319] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:39.483616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.599470 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7319-14f5-45e5-ae34-e1fa76eacc0f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.605453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.606663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.606949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.607260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.608213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.610447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.649013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1147/2320] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:39 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:44:39.652908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.667935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bc63b-4279-41dd-bfc6-b44ad93a08dc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.826893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.830796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.831275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.831708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.834757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.836828 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.902785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.909864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1174/2321] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:39 2022] POST /identity/v3/users => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:39.919583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147d87a-8b17-4b3d-95de-d2f0dc51b652 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:39.933681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:39.935598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:39.935870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:39.936151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:39.938209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:39.954462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:39.979088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:39.989135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1148/2322] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:44:39.989997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:39.993841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a3266-4fb8-4bc3-9a96-7701d49bc0e1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:40.010039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:40.010891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:40.011249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:40.015612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] PATH_INFO: `/v3/projects/1576a489c15e4b89bd1283872ec594d2/users/8023712f65f54f0ba8e5a16e40a78e14/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:40.047554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1576a489c15e4b89bd1283872ec594d2, user_id=8023712f65f54f0ba8e5a16e40a78e14, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:40.050639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:40.090070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:40.095214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1175/2323] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:39 2022] PUT /identity/v3/projects/1576a489c15e4b89bd1283872ec594d2/users/8023712f65f54f0ba8e5a16e40a78e14/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:40.103069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a2ea-4596-477d-a816-976317ee8460 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:40.242939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:40.243677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:40.243961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:40.244272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:40.245124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:40.246790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8f010-023a-4fae-a54c-113463fd8eac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:40.262036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1149/2324] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:40.263978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:40.276933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975cfe2-7e90-40d9-8c9c-d92ca8d78365 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:40.283442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:40.284730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:40.285018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:40.285294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] PATH_INFO: `/v3/projects/1576a489c15e4b89bd1283872ec594d2/users/8023712f65f54f0ba8e5a16e40a78e14/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:40.287434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1576a489c15e4b89bd1283872ec594d2, user_id=8023712f65f54f0ba8e5a16e40a78e14, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:40.290543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:40.303283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1176/2325] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:40 2022] PUT /identity/v3/projects/1576a489c15e4b89bd1283872ec594d2/users/8023712f65f54f0ba8e5a16e40a78e14/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:40.314878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:40.315209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:40.315958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:40.343749 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:40.370358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] MFA Rules not processed for user `8023712f65f54f0ba8e5a16e40a78e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:40.371692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:40.536780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1150/2326] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:40.552198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:40.564012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:40.749369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34786e3b-9341-4953-991e-9cb948ae6f03 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:40.753448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:40.754195 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:40.754471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:40.754901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:40.792010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:40.801081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2ece-ec33-4536-b127-ff90ac57ef69 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:40.824414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1177/2327] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:40.969094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:40.983831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25878f-337b-4a3b-9b3f-9fce4233e70d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:40.994896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:41.002456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:41.003142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:41.003639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:41.238871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:41.241850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:41.293342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1151/2328] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:46.377928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:46.378457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:46.378795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:46.446531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] MFA Rules not processed for user `1636311c73b9461c9a64db18e8fdf0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:46.448189 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:46.546161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1178/2329] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:44:46.553823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:46.555431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:46.661339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eebcf8-7849-4b5a-af3b-b254822c1d85 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:46.669362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:46.670272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:46.670558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:46.670841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:46.671581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:46.673924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:46.732651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1152/2330] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:46 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:46.736757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:46.769201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d9336-3ba0-410c-8955-dee26fe7ba8f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:46.838101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:46.846242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:46.847043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:46.851113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:46.851113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:46.851113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:46.853309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:46.936823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1179/2331] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:46 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:46.937008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:46.950535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbef6f4-11ee-487e-91e2-76f083f26f87 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:46.962656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:46.963604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:46.964954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:46.965179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:46.965559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:46.966421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:46.968541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:46.979016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029fd28-2fe5-4ccd-b94d-c1f3d2f6f827 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:46.986086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:46.987221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:46.987658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:46.988104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] PATH_INFO: `/v3/users/195d647108764778b5779b7225b413fa` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:46.989550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=195d647108764778b5779b7225b413fa)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:46.991348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1153/2332] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:46.992247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:46.993068 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:47.005149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:47.019997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e71aa7-c237-4715-a2ce-6dfffedb12bd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:47.031573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:47.033482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:47.033872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:47.034829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] PATH_INFO: `/v3/projects/98bea12852fc45879f303aa81b795485/users/a1d741ae3bf1446eb81d03d93b673830/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:47.071235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98bea12852fc45879f303aa81b795485, user_id=a1d741ae3bf1446eb81d03d93b673830, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:47.081226 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:47.083600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '195d647108764778b5779b7225b413fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:47.108521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '195d647108764778b5779b7225b413fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:47.109923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '195d647108764778b5779b7225b413fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:47.118607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1154/2333] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:46 2022] PUT /identity/v3/projects/98bea12852fc45879f303aa81b795485/users/a1d741ae3bf1446eb81d03d93b673830/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:47.120043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:47.126779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '195d647108764778b5779b7225b413fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:47.136516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-66b4b222-9bc0-4929-a8bd-9299b80dda16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '195d647108764778b5779b7225b413fa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:47.144491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1180/2334] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:46 2022] DELETE /identity/v3/users/195d647108764778b5779b7225b413fa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:47.157422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:47.157538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:47.157943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:47.201357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:47.203700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:47.239850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a516-5261-416e-8df3-34b2b1d4480e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:47.339098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1181/2335] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:47.354584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:47.358701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:47.395014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:47.396557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:47.396854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:47.397136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:47.400108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:47.404491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:47.434270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1155/2336] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:47.438366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:47.443064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:47.482186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a2cc-eb37-4811-b6c5-68c344d427c5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:47.487557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827caac-df6c-43d4-bc58-547a23014b1d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:47.494176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:47.494882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:47.495204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:47.495987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:47.500077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:47.502025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:47.502429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:47.502814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] PATH_INFO: `/v3/projects/98bea12852fc45879f303aa81b795485/users/a1d741ae3bf1446eb81d03d93b673830/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:47.505218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98bea12852fc45879f303aa81b795485, user_id=a1d741ae3bf1446eb81d03d93b673830, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:47.509241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:47.512762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:47.514260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:47.524664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1156/2337] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:47 2022] PUT /identity/v3/projects/98bea12852fc45879f303aa81b795485/users/a1d741ae3bf1446eb81d03d93b673830/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:47.540209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1182/2338] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:48.160724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:48.163410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:48.339257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73729-ceac-42e6-bc03-01be007f2680 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:48.355977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:48.357102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:48.357404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:48.357687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] PATH_INFO: `/v3/projects/e289ee76ed784a82bb7cbfa6a0cc471f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:48.359061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e289ee76ed784a82bb7cbfa6a0cc471f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:48.361295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:48.387274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:48.468744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e289ee76ed784a82bb7cbfa6a0cc471f', 'request_id': 'req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.488301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e289ee76ed784a82bb7cbfa6a0cc471f', 'request_id': 'req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.495848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e289ee76ed784a82bb7cbfa6a0cc471f', 'request_id': 'req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.526809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] The token cache is being invalidate because project e289ee76ed784a82bb7cbfa6a0cc471f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:48.527216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.528346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.530619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1157/2339] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:48 2022] DELETE /identity/v3/projects/e289ee76ed784a82bb7cbfa6a0cc471f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:48.535180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:48.541647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:48.554540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b9517-f9f0-45a4-91f7-b3cfab532dd0 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:48.691099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:48.692424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:48.692946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:48.693403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] PATH_INFO: `/v3/users/8023712f65f54f0ba8e5a16e40a78e14` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:48.698851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8023712f65f54f0ba8e5a16e40a78e14)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:48.701344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:48.782424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8023712f65f54f0ba8e5a16e40a78e14'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.808750 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8023712f65f54f0ba8e5a16e40a78e14'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.809617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8023712f65f54f0ba8e5a16e40a78e14'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.822618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8023712f65f54f0ba8e5a16e40a78e14'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.837048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1b18cc7-8600-472d-90f3-95f463046080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8023712f65f54f0ba8e5a16e40a78e14'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:48.848154 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1183/2340] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:44:48 2022] DELETE /identity/v3/users/8023712f65f54f0ba8e5a16e40a78e14 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:50.228291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:50.229816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:50.254097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e98f3d-b073-4f04-9f5e-76e1e4c2ea69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:50.441696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:50.445021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:50.445444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:50.445743 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] PATH_INFO: `/v3/projects/1576a489c15e4b89bd1283872ec594d2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:50.451107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1576a489c15e4b89bd1283872ec594d2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:50.457461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:50.649194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1576a489c15e4b89bd1283872ec594d2', 'request_id': 'req-0b0e59bd-008d-438f-a719-a2abb2709ab7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:50.664988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1576a489c15e4b89bd1283872ec594d2', 'request_id': 'req-0b0e59bd-008d-438f-a719-a2abb2709ab7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:50.685965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1576a489c15e4b89bd1283872ec594d2', 'request_id': 'req-0b0e59bd-008d-438f-a719-a2abb2709ab7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:50.713011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] The token cache is being invalidate because project 1576a489c15e4b89bd1283872ec594d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:44:50.713434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:50.714298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0b0e59bd-008d-438f-a719-a2abb2709ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:44:50.716911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1158/2341] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:44:50 2022] DELETE /identity/v3/projects/1576a489c15e4b89bd1283872ec594d2 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:44:50.722636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04513c30-8c48-4434-89b2-447164427451 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:50.723200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04513c30-8c48-4434-89b2-447164427451 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:50.723469 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04513c30-8c48-4434-89b2-447164427451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:50.748174 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-04513c30-8c48-4434-89b2-447164427451 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:50.749397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-04513c30-8c48-4434-89b2-447164427451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:50.889554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1184/2342] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:50.900631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:50.900975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:50.901290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:50.932059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:50.933889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:50.952513 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04513c30-8c48-4434-89b2-447164427451 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:50.959803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1159/2343] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:50.966587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04513c30-8c48-4434-89b2-447164427451 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:50.972406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:50.973526 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:50.974033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:50.974462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:50.975614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:44:50.976068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:50.978273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.000706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1185/2344] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:44:51.002344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.013084 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2afaf6-bf0c-4107-950f-28bf3855e280 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:51.018385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:51.019348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.019801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.020229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.020946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:51.027655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.072470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.080094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1160/2345] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:44:51 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:44:51.087350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bd9a3-62c2-4334-91f0-b8d1ddc826e7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:51.101030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:51.287573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:51.288453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.291412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.291869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.292676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:51.296309 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.387444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1186/2346] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:44:51 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:44:51.391623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.399638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:51.412327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd415eb-5c86-4b50-abfe-b3a6de2f4e1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:51.430111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:51.430796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.430969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.431305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.432059 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:51.433680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.470175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1161/2347] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:44:51.471312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.493330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e10d93-f5d0-4564-9872-d28a82cbb147 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:51.499791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:51.500856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.501229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.501532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.502472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:51.504094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc23df6-d974-44eb-991f-42e89abab233 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.538631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1187/2348] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:44:51.540010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.554343 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbef73-6fdd-43cb-a009-92ef23e80a87 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:51.579000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:51.579970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.580249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.580523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] PATH_INFO: `/v3/projects/d2a0871a3361404aa42a08fe0145ae29/users/daf320646e6343b7afbf5ed14e0c105d/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.645035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a0871a3361404aa42a08fe0145ae29, user_id=daf320646e6343b7afbf5ed14e0c105d, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:51.658344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:51.712768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1162/2349] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:44:51 2022] PUT /identity/v3/projects/d2a0871a3361404aa42a08fe0145ae29/users/daf320646e6343b7afbf5ed14e0c105d/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:44:51.727673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:51.728515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:51.728798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:51.761209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] MFA Rules not processed for user `daf320646e6343b7afbf5ed14e0c105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:51.762612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:51.943532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1188/2350] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:51.951168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:51.953511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:52.124618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d082-a261-4cce-b52a-7bbe5d283a15 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:52.130869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:52.132573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:52.133040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:52.133469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:52.147591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:52.149526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f5ae6-24d7-4272-bb8e-f1da2c14b686 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:52.164078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1163/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:53.961863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:53.966471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:54.240258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:54.253845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ff80d-a5d3-49a2-bc8a-856982dd9e11 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:54.261557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:54.262880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:54.265113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:54.265565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:54.267913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:54.284731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:54.288605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:54.322276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1189/2352] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:55.719530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a013-c8a0-49c5-98db-447713e49828 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:55.720001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a013-c8a0-49c5-98db-447713e49828 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:55.720091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9a013-c8a0-49c5-98db-447713e49828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:55.751786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6ab9a013-c8a0-49c5-98db-447713e49828 None None] MFA Rules not processed for user `a1d741ae3bf1446eb81d03d93b673830`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:55.753138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6ab9a013-c8a0-49c5-98db-447713e49828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:55.855292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1164/2353] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:44:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:55.865255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:55.866802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:55.880829 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f6338-2046-41e0-b638-985278a68af3 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:55.885787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:55.886680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:55.887358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:55.887795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:55.889724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:55.968253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:55.969953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:55.984464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1190/2354] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:44:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:44:56.120096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb07c1-5db8-4449-8a01-089ce053a6e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:56.120418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb07c1-5db8-4449-8a01-089ce053a6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:56.120654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb07c1-5db8-4449-8a01-089ce053a6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:56.166321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dddb07c1-5db8-4449-8a01-089ce053a6e0 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:44:56.168656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dddb07c1-5db8-4449-8a01-089ce053a6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:56.232345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:44:56.252147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1165/2355] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:44:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:44:56.265215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:44:56.267149 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:44:56.369639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b301c96-5bb0-428a-8df8-d2245c2bf9b3 service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:44:56.375449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:44:56.376202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:44:56.376693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:44:56.376975 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:44:56.388524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:44:56.394257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d14c8-9eb5-4817-8cd1-3488a5d31da0 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:44:56.402390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1191/2356] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:44:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:45:03.930732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a76d-a97f-4cc7-8fd5-430f6f1b0d0a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:03.931879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a76d-a97f-4cc7-8fd5-430f6f1b0d0a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:03.932865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69a76d-a97f-4cc7-8fd5-430f6f1b0d0a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:03.934912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1166/2357] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:45:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:45:03.945987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594df2c-2d3e-41f1-9478-d34e25aa25d2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:03.946558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594df2c-2d3e-41f1-9478-d34e25aa25d2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:03.946558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594df2c-2d3e-41f1-9478-d34e25aa25d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:04.035988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c594df2c-2d3e-41f1-9478-d34e25aa25d2 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:04.037252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c594df2c-2d3e-41f1-9478-d34e25aa25d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:04.124774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:04.183816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1192/2358] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:45:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:06.245716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:06.961591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69a76d-a97f-4cc7-8fd5-430f6f1b0d0a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:06.985635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69a76d-a97f-4cc7-8fd5-430f6f1b0d0a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:06.989409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:06.990066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:06.990355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:06.990634 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.001249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:07.002726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e50f2c-dec2-4282-a2c4-0ae24f737768 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:07.016256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1167/2359] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:07.023934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765e444-117f-481b-b28b-9a15d2df9ea4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.024848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765e444-117f-481b-b28b-9a15d2df9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.025127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765e444-117f-481b-b28b-9a15d2df9ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.042856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b765e444-117f-481b-b28b-9a15d2df9ea4 None None] MFA Rules not processed for user `1636311c73b9461c9a64db18e8fdf0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:07.044102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b765e444-117f-481b-b28b-9a15d2df9ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:07.055430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1193/2360] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:07.088358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.088719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.089020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.090753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1168/2361] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:45:07.096495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2de6-99b8-4fd4-baea-2cc18231d615 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.096837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2de6-99b8-4fd4-baea-2cc18231d615 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.097061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2de6-99b8-4fd4-baea-2cc18231d615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.119311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a4eb2de6-99b8-4fd4-baea-2cc18231d615 None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:07.120507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a4eb2de6-99b8-4fd4-baea-2cc18231d615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:07.198889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1194/2362] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:45:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:07.216295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:07.218451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:07.236453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acaf2-af56-428c-8aa9-66f2c66f5ac5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:07.242473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:07.243480 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.244452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.245137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.256186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:07.258043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046f006-3310-41d2-a7c1-1aeddc565184 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:07.269796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1169/2363] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:07.362698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d67c-4c90-45f4-a82d-ad028c41da3e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.363111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d67c-4c90-45f4-a82d-ad028c41da3e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.363406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2d67c-4c90-45f4-a82d-ad028c41da3e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.364862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1195/2364] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:45:07.370594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed9d7-6ced-42e5-95b1-2c2aa783b7af None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.370886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed9d7-6ced-42e5-95b1-2c2aa783b7af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.371264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed9d7-6ced-42e5-95b1-2c2aa783b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.391709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a50ed9d7-6ced-42e5-95b1-2c2aa783b7af None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:07.392924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a50ed9d7-6ced-42e5-95b1-2c2aa783b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:07.402846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1170/2365] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:45:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:07.413843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2d67c-4c90-45f4-a82d-ad028c41da3e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:07.425038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2d67c-4c90-45f4-a82d-ad028c41da3e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:07.429236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:07.429933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:07.430218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:07.430481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:07.441433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:07.442925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:07.454746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1196/2366] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:10.326432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:10.327445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:10.327924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:10.430096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:10.431525 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:10.440262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:10.442566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:10.457484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1171/2367] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:45:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:10.473452 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:10.515357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdd343-1921-4105-8ac3-73fbedcf95a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:10.522131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:10.523116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:10.523412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:10.523692 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:10.532599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:10.534482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd6c2c-ae9f-4c19-b829-127f758b3617 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:10.539172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc93bd-5a33-438e-aed1-273dde3493e2 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:10.543615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1172/2368] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:45:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 09:45:10.543917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:10.544895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:10.545531 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:10.546029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] PATH_INFO: `/v3/users/b4f77464774649d5bab2c94f612ea435` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:10.548435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f77464774649d5bab2c94f612ea435)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:10.550735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:10.636313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f77464774649d5bab2c94f612ea435'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:10.651448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f77464774649d5bab2c94f612ea435'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:10.652457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f77464774649d5bab2c94f612ea435'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:10.656318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:10.665250 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f77464774649d5bab2c94f612ea435'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:10.681648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f77464774649d5bab2c94f612ea435'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:10.694894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1197/2369] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:45:10 2022] DELETE /identity/v3/users/b4f77464774649d5bab2c94f612ea435 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:10.699199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:10.699555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:10.699845 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:10.723701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:10.725770 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:10.840803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1173/2370] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:10.853160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:10.855265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:10.989783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842233-6c29-438f-a4f3-b9081c8093f1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:10.996201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:10.997276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:10.997841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:10.998319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:11.014787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:11.016572 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185553b-4e17-4b10-90f9-accd33d4f4e4 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:11.040027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1198/2371] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:11.666438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:11.667982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:11.681672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12405152-e42a-4dea-9bb9-b7a66a878061 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:11.689330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:11.690632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:11.691202 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:11.691687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] PATH_INFO: `/v3/projects/d0396f3f4a90429fa40065d815ac1aa5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:11.693168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0396f3f4a90429fa40065d815ac1aa5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:11.695551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:11.785188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0396f3f4a90429fa40065d815ac1aa5', 'request_id': 'req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:11.791022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:11.824881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0396f3f4a90429fa40065d815ac1aa5', 'request_id': 'req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:11.833414 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0396f3f4a90429fa40065d815ac1aa5', 'request_id': 'req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:11.845751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] The token cache is being invalidate because project d0396f3f4a90429fa40065d815ac1aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:45:11.846147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:11.846818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:11.848442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1174/2372] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:45:11 2022] DELETE /identity/v3/projects/d0396f3f4a90429fa40065d815ac1aa5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:15.947445 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e45db3-fdda-494d-9ada-ec8edcfcc239 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:15.948084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e45db3-fdda-494d-9ada-ec8edcfcc239 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:15.948084 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e45db3-fdda-494d-9ada-ec8edcfcc239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:15.966672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c6e45db3-fdda-494d-9ada-ec8edcfcc239 None None] MFA Rules not processed for user `7f8df9faa4dd4eeeb724e891cdb55616`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:15.967981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c6e45db3-fdda-494d-9ada-ec8edcfcc239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:16.037147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1199/2373] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:21.842303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:22.250208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:22.251860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:22.324080 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dfb5f-a633-4fb3-9cc8-1a5dbdc55685 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:22.329431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:22.330267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:22.330598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:22.330834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] PATH_INFO: `/v3/users/daf320646e6343b7afbf5ed14e0c105d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:22.332120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf320646e6343b7afbf5ed14e0c105d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:22.334136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:22.399953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf320646e6343b7afbf5ed14e0c105d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:22.407499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf320646e6343b7afbf5ed14e0c105d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:22.415936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf320646e6343b7afbf5ed14e0c105d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:22.416642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf320646e6343b7afbf5ed14e0c105d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:22.417262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf320646e6343b7afbf5ed14e0c105d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:22.438610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1175/2374] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:45:22 2022] DELETE /identity/v3/users/daf320646e6343b7afbf5ed14e0c105d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:22.443640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa748ca-76db-4640-ab87-77137eb8a44d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:22.444170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa748ca-76db-4640-ab87-77137eb8a44d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:22.444310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa748ca-76db-4640-ab87-77137eb8a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:22.464008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-faa748ca-76db-4640-ab87-77137eb8a44d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:22.465382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-faa748ca-76db-4640-ab87-77137eb8a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:22.535841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1200/2375] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:22.545842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:22.547382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:22.630461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93eb29-c771-4d84-b9e2-a3749f047c59 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:22.637212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:22.638063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:22.638422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:22.638791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:22.655381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:22.657540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:22.673069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1176/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:45:23.224882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.225417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.225774 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.253650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:23.255624 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.287096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:23.321739 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.322879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1201/2377] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:23.324806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.336296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.337856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.340322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962604-7f1d-4ffe-813f-9dfa4c2ea700 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.345574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.346453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.346768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.347048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] PATH_INFO: `/v3/projects/d2a0871a3361404aa42a08fe0145ae29` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.348736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a0871a3361404aa42a08fe0145ae29)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.350713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.368763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:23.436561 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77f11a-68fc-4afc-a685-eacf51753e01 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.443983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a0871a3361404aa42a08fe0145ae29', 'request_id': 'req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:23.454441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a0871a3361404aa42a08fe0145ae29', 'request_id': 'req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:23.461040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a0871a3361404aa42a08fe0145ae29', 'request_id': 'req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:23.475250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] The token cache is being invalidate because project d2a0871a3361404aa42a08fe0145ae29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:45:23.475534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:23.476517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba0037a1-9d61-4e27-b512-b3e5b17e62b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:23.478364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1177/2378] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:45:23 2022] DELETE /identity/v3/projects/d2a0871a3361404aa42a08fe0145ae29 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:45:23.483572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3793ce-aa3e-40fe-8faf-3fc853036177 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.483870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3793ce-aa3e-40fe-8faf-3fc853036177 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.484197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3793ce-aa3e-40fe-8faf-3fc853036177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.508520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ab3793ce-aa3e-40fe-8faf-3fc853036177 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:23.510078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ab3793ce-aa3e-40fe-8faf-3fc853036177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.516996 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.517763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.518117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.518460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.520809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.583268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1178/2379] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:45:23.588361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.588658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.588958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.590101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.592354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.604044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1202/2380] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:45:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 09:45:23.617938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:23.619318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:23.629155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1179/2381] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:45:23.633676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.644917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac8ee-b2df-475e-a364-ca75496bdc6e service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.656630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.658639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.659417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.660218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.660815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:45:23.661194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.663255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.667291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:23.693002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.703132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1203/2382] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:45:23.705766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fc6b4-dc47-4a21-9350-5f233ecb5ea4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.713451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.714152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.714443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.714720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.715492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.718100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.763629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1180/2383] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:45:23 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:45:23.763812 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.774993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939570a7-0ace-4a04-93a0-ee3794af7add admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.840341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.841759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.842450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.842450 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.843232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.845788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.884180 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.895785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1204/2384] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:45:23 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:45:23.902632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02d89a-506b-4d6d-9604-b7efd69f998b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.917293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.917293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.917293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.917293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.917714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.918982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:23.940093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1181/2385] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:45:23.941398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:23.952295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebabfc-204e-4e44-9085-b24102e50ae5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:23.970457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:23.971289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:23.971605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:23.971966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:23.972876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:23.976244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104174a6-f904-4e8c-bfb0-25e7ee86776d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:24.008100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1205/2386] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:45:24.009808 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:24.035381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83a658-8e43-4ccf-b972-d96014bce8d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:24.056120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:24.057535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:24.058030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:24.058532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] PATH_INFO: `/v3/projects/5cda5f0fca494a47b84b563e54809229/users/5eaefffc2e5a4a339fd3c2c5d98b515d/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:24.097121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cda5f0fca494a47b84b563e54809229, user_id=5eaefffc2e5a4a339fd3c2c5d98b515d, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:24.100164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:24.126497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1182/2387] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:45:24 2022] PUT /identity/v3/projects/5cda5f0fca494a47b84b563e54809229/users/5eaefffc2e5a4a339fd3c2c5d98b515d/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:24.137239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:24.137530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:24.137834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:24.198908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] MFA Rules not processed for user `5eaefffc2e5a4a339fd3c2c5d98b515d`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:24.200327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:24.374914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1206/2388] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:24.375280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:24.490393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:24.524965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df206b-8fbd-4120-80d5-00e1bee8ddf1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:24.534607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:24.535706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:24.536224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:24.536902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:24.539565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:24.543156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:24.585669 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1183/2389] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:45:24 2022] POST /identity/v3/projects => generated 375 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:45:24.589980 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:24.609769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a08e7-94a3-4052-8eb9-128e9f35d3db None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:24.796538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:24.797615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:24.798088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:24.798528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:24.800638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:24.806332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:24.826743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:24.857351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:24.860938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1207/2390] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:45:24 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:45:24.972987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a197-5678-401c-a573-84ced8499db4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:24.978206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:24.978911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:24.979244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:24.979526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:24.980272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:24.984154 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:25.019066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1184/2391] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:45:25.023507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:25.045800 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea001ee-a229-4822-90fc-b73db304c448 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:25.051203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:25.052105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:25.052394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:25.052676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] PATH_INFO: `/v3/projects/dcfb91098f5f45a6866c56c75448936a/users/e84df3aa97df4649b8d061b4efd35fc6/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:25.095678 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfb91098f5f45a6866c56c75448936a, user_id=e84df3aa97df4649b8d061b4efd35fc6, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:25.098819 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:25.141960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1208/2392] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:45:25 2022] PUT /identity/v3/projects/dcfb91098f5f45a6866c56c75448936a/users/e84df3aa97df4649b8d061b4efd35fc6/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:25.146044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:25.158243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7c368-95db-4fe6-b5c8-a8a295f414ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:25.346947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:25.347804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:25.348316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:25.348696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:25.349539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:25.351663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b07115-021a-49a3-9f02-812fe10a9d14 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:25.375814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1185/2393] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:45:25.380592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:25.401032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c946ea-e587-40e2-8932-d09334aadedd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:25.415233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:25.416169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:25.416447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:25.416728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] PATH_INFO: `/v3/projects/dcfb91098f5f45a6866c56c75448936a/users/e84df3aa97df4649b8d061b4efd35fc6/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:25.419625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfb91098f5f45a6866c56c75448936a, user_id=e84df3aa97df4649b8d061b4efd35fc6, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:25.422299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:25.447681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1209/2394] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:45:25 2022] PUT /identity/v3/projects/dcfb91098f5f45a6866c56c75448936a/users/e84df3aa97df4649b8d061b4efd35fc6/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:25.458293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:25.458659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:25.458857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:25.503531 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] MFA Rules not processed for user `e84df3aa97df4649b8d061b4efd35fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:25.505088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:25.649995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1186/2395] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:25.663314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:25.665798 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:25.827161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbf279-d18d-4489-8a2c-9dcf5a32b14e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:25.844990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:25.845682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:25.845970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:25.846258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:25.983724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:25.985213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94bbfb-fab2-494c-90af-3762afb27382 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:25.995896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1210/2396] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:28.806014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:28.857281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:28.901773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f263-53a3-43de-89f4-a54cda170834 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:28.906323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:28.907445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:28.907823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:28.908171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] PATH_INFO: `/v3/users/5eaefffc2e5a4a339fd3c2c5d98b515d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:28.909477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eaefffc2e5a4a339fd3c2c5d98b515d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:28.911703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:28.971490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eaefffc2e5a4a339fd3c2c5d98b515d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:28.981207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eaefffc2e5a4a339fd3c2c5d98b515d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:28.991028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eaefffc2e5a4a339fd3c2c5d98b515d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:28.991701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eaefffc2e5a4a339fd3c2c5d98b515d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:28.992556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eaefffc2e5a4a339fd3c2c5d98b515d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.010631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1187/2397] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:45:28 2022] DELETE /identity/v3/users/5eaefffc2e5a4a339fd3c2c5d98b515d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:29.015315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:29.015857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:29.015970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:29.032937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:29.034150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:29.086234 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:29.101943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1211/2398] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:29.112579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:29.114595 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:29.177713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5e1f1-9a65-4214-b879-5e1061b519e2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:29.181603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:29.182275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:29.182563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:29.182841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:29.193523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:29.194974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:29.204429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1188/2399] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:29.730363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:29.732051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:29.744889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382f53d-394b-40cc-995e-2ba2c0c7823d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:29.759496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:29.760411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:29.760694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:29.760981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] PATH_INFO: `/v3/projects/5cda5f0fca494a47b84b563e54809229` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:29.762290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cda5f0fca494a47b84b563e54809229)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:29.770167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:29.869785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cda5f0fca494a47b84b563e54809229', 'request_id': 'req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.880518 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cda5f0fca494a47b84b563e54809229', 'request_id': 'req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.900371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cda5f0fca494a47b84b563e54809229', 'request_id': 'req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.919984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] The token cache is being invalidate because project 5cda5f0fca494a47b84b563e54809229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:45:29.920259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.924905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:29.926804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1212/2400] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:45:29 2022] DELETE /identity/v3/projects/5cda5f0fca494a47b84b563e54809229 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:29.931767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:29.944548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f7938-bd93-4f32-a399-62426e936051 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:30.010006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:30.010902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.011224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.011505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] PATH_INFO: `/v3/users/e84df3aa97df4649b8d061b4efd35fc6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.012831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84df3aa97df4649b8d061b4efd35fc6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:30.014780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:30.016382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:30.075000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84df3aa97df4649b8d061b4efd35fc6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.083013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84df3aa97df4649b8d061b4efd35fc6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.092040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84df3aa97df4649b8d061b4efd35fc6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.092705 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84df3aa97df4649b8d061b4efd35fc6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.093413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-65fe63bb-d7bf-41fa-85d1-404297a657ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84df3aa97df4649b8d061b4efd35fc6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.117302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1189/2401] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:45:29 2022] DELETE /identity/v3/users/e84df3aa97df4649b8d061b4efd35fc6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:30.453359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:30.454895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:30.470092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9f01-8304-43bc-a822-9db4e4ce85ca admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:30.534002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:30.535338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.535725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.536229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] PATH_INFO: `/v3/projects/dcfb91098f5f45a6866c56c75448936a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.537588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcfb91098f5f45a6866c56c75448936a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:30.539728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:30.603227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcfb91098f5f45a6866c56c75448936a', 'request_id': 'req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.609539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcfb91098f5f45a6866c56c75448936a', 'request_id': 'req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.619899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcfb91098f5f45a6866c56c75448936a', 'request_id': 'req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.632002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] The token cache is being invalidate because project dcfb91098f5f45a6866c56c75448936a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:45:30.632002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.632541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c0cf99b0-7939-4364-91a8-fa12bf0bc9ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:45:30.634374 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1213/2402] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:45:30 2022] DELETE /identity/v3/projects/dcfb91098f5f45a6866c56c75448936a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:30.639026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.639352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.639631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.658726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:30.660056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:30.760169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1190/2403] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:30.770888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.771056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.771597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.787676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:30.799876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:30.802312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:30.813483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1214/2404] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:30.823004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:30.832322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8eec4-7d03-4557-a7d6-61910ad80c66 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:30.837599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:30.838375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.838668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.839027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.839899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:45:30.840293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:30.851901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:30.879858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1191/2405] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:45:30.884385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:30.895511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa2e41-3396-46af-a01c-eff53ebb0f2c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:30.911866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:30.912559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:30.912842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:30.913118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:30.913818 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:30.915961 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:30.948105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1215/2406] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:45:30 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:45:30.949315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:30.959280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834999fd-e894-412b-89d3-320a233237c2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:31.039476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:31.040302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.040652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.040993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.041763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:31.044200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:31.057589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:31.083126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1192/2407] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:45:30 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:45:31.087999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:31.104121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0c52-be18-4456-a048-25564b580027 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:31.109042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:31.109731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.110048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.110336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.111093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:31.112733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:31.125468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1216/2408] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:45:31.129906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:31.147545 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f256de-cb76-4f9b-ae19-f0039e7b2f3c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:31.154035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:31.154744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.155030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.155345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.156289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:31.158883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456aaa6-e871-46bb-b13a-68df88d72eea admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:31.172081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1193/2409] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:45:31.176067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:31.190749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b055fe-08b0-4919-a597-8906a1397176 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:31.195997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:31.197151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.197605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.198058 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] PATH_INFO: `/v3/projects/c15e1bb5bd1741ae98304f809dea24f7/users/a3283218fb79487491de597fdfc16916/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.216148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15e1bb5bd1741ae98304f809dea24f7, user_id=a3283218fb79487491de597fdfc16916, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:31.218846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:31.241636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1217/2410] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:45:31 2022] PUT /identity/v3/projects/c15e1bb5bd1741ae98304f809dea24f7/users/a3283218fb79487491de597fdfc16916/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:45:31.251933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941eb0-7fba-42fe-ae98-a3441b797ad8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.252299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941eb0-7fba-42fe-ae98-a3441b797ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.252659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b941eb0-7fba-42fe-ae98-a3441b797ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.321036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3b941eb0-7fba-42fe-ae98-a3441b797ad8 None None] MFA Rules not processed for user `a3283218fb79487491de597fdfc16916`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:31.323799 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b941eb0-7fba-42fe-ae98-a3441b797ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:31.392765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1194/2411] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:31.403891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:31.405936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:31.476926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77636d-f100-47ec-aca7-5c98661d6009 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:31.480741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:31.481403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:31.481682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:31.481955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:31.491742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:31.493191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3873a3-3898-4c5a-acc4-aa9ed20ab22b service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:31.502086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1218/2412] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:32.938999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:32.939631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:32.940043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:32.959186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] MFA Rules not processed for user `a3283218fb79487491de597fdfc16916`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:32.960402 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:32.968109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1195/2413] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:41.498042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:42.953638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=42 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:47.803360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66ce8-4ac0-43e1-9e8a-171f2051e344 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:47.803935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66ce8-4ac0-43e1-9e8a-171f2051e344 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:47.803935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66ce8-4ac0-43e1-9e8a-171f2051e344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:47.826954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b7c66ce8-4ac0-43e1-9e8a-171f2051e344 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:47.828231 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b7c66ce8-4ac0-43e1-9e8a-171f2051e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:47.889997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1219/2414] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:47.908057 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:47.909468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:47.994495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ab0fa-8d1a-483f-a73d-8d429744e9c0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:48.005642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:48.006417 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:48.006954 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:48.007357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:48.023904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:48.025934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:48.037384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1196/2415] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:45:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 09:45:57.882304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=25 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:58.032623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=32 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:45:58.319658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20271f2-de3b-4236-a79a-a3580993252d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:58.319994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20271f2-de3b-4236-a79a-a3580993252d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:58.320284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20271f2-de3b-4236-a79a-a3580993252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:58.339334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f20271f2-de3b-4236-a79a-a3580993252d None None] MFA Rules not processed for user `fd136fcda9a04a16b258301a86bde8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:58.340557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f20271f2-de3b-4236-a79a-a3580993252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:58.399905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1220/2416] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:59.562736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:59.564342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:59.645697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce56f-181d-4b84-87af-ab77b8e032de service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:59.652445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:59.653774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:59.654222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:59.654691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:59.656796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:59.769604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:59.771410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:59.789915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1197/2417] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:45:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:45:59.836502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53a3-1749-490a-80b2-2ec72556a425 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:59.839443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53a3-1749-490a-80b2-2ec72556a425 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:59.839790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53a3-1749-490a-80b2-2ec72556a425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:59.861404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f7ef53a3-1749-490a-80b2-2ec72556a425 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:45:59.862883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f7ef53a3-1749-490a-80b2-2ec72556a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:45:59.870055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1221/2418] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:45:59.896395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:45:59.913741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b72ba7-4f7a-4943-b171-ffb309d48437 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:45:59.919054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:45:59.920746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:45:59.920746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:45:59.920746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:45:59.929760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:45:59.931734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:45:59.955021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1198/2419] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:45:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:46:00.573785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb63e0-22bf-4435-a003-b6a12b4a62d9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:00.574198 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb63e0-22bf-4435-a003-b6a12b4a62d9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:00.574530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb63e0-22bf-4435-a003-b6a12b4a62d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:00.594447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e1bb63e0-22bf-4435-a003-b6a12b4a62d9 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:00.596080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e1bb63e0-22bf-4435-a003-b6a12b4a62d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:00.605359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1222/2420] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:46:00.620086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:00.636309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5dd5b-1f51-4b7f-bc8d-614fce24d5be service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:00.643435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:00.644374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:00.644752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:00.645123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:00.656269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:00.658021 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:00.668976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1199/2421] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:46:00.896362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b790-cd65-43ad-b5ec-0abde91c06a1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:00.896901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b790-cd65-43ad-b5ec-0abde91c06a1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:00.896901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b790-cd65-43ad-b5ec-0abde91c06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:00.917119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e7e5b790-cd65-43ad-b5ec-0abde91c06a1 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:00.918473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7e5b790-cd65-43ad-b5ec-0abde91c06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:00.927320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1223/2422] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:00.942499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:00.954840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd175b-5539-455f-911a-761c2d759715 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:00.963262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:00.963953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:00.964690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:00.965137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:00.976035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:00.977715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:00.987732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1200/2423] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:46:01.382078 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:01.382368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:01.382646 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:01.402094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:01.403353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:01.412005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1224/2424] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:01.429415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:01.444624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d4f32-2485-43ae-97e8-fa6919218b55 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:01.453606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:01.454638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:01.455209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:01.455843 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:01.465959 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:01.469245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e047f5d-1629-4704-9cb1-e73273f3d213 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:01.482986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1201/2425] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:46:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:46:04.242449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:04.306030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713cccfb-f7b9-4b82-bf84-652167888297 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:04.311394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:04.312356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:04.312698 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:04.312981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] PATH_INFO: `/v3/users/a3283218fb79487491de597fdfc16916` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:04.314455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3283218fb79487491de597fdfc16916)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:04.316458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:04.369794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3283218fb79487491de597fdfc16916'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:04.382677 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3283218fb79487491de597fdfc16916'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:04.383403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3283218fb79487491de597fdfc16916'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:04.392142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3283218fb79487491de597fdfc16916'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:04.399578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3283218fb79487491de597fdfc16916'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:04.401155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:04.408044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1225/2426] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:46:04 2022] DELETE /identity/v3/users/a3283218fb79487491de597fdfc16916 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:46:04.412824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:04.413305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:04.413397 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:04.432131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:04.433493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:04.491013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:04.506018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1202/2427] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:46:04.521621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:04.523177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:04.668455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974a184-6f86-4a28-b1bc-6bfd6a290d55 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:04.679730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:04.680732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:04.681215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:04.681645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:04.698722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:04.702268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533ed9c-189d-417c-a9d5-f88f27de74e2 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:04.726784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1226/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:04.940361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:04.943509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:04.963209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb16b8-3de6-4d8b-9772-7af79c8148ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:04.969333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:04.970569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:04.971011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:04.971679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] PATH_INFO: `/v3/projects/c15e1bb5bd1741ae98304f809dea24f7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:04.973072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15e1bb5bd1741ae98304f809dea24f7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:04.975617 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:05.040004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15e1bb5bd1741ae98304f809dea24f7', 'request_id': 'req-c436297f-94f7-4b5b-9f9f-c9c03ae30662'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:05.049575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15e1bb5bd1741ae98304f809dea24f7', 'request_id': 'req-c436297f-94f7-4b5b-9f9f-c9c03ae30662'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:05.058904 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15e1bb5bd1741ae98304f809dea24f7', 'request_id': 'req-c436297f-94f7-4b5b-9f9f-c9c03ae30662'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:05.074805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] The token cache is being invalidate because project c15e1bb5bd1741ae98304f809dea24f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:46:05.075529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:05.076554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c436297f-94f7-4b5b-9f9f-c9c03ae30662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:05.078711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1203/2429] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:46:04 2022] DELETE /identity/v3/projects/c15e1bb5bd1741ae98304f809dea24f7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:46:14.718960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=27 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:15.070637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:21.910177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:21.910788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:21.910788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:21.933447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] MFA Rules not processed for user `d5e6d9fad06449c1b43254df442d224f`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:21.936311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:21.999760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1227/2430] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:31.988806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=25 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:47.386640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:47.387184 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:47.387287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:47.405119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] MFA Rules not processed for user `544cf118cc9e4aed8e7ec7e9dfe023bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:47.406328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:47.461250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1204/2431] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:47.501004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:47.502617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:47.584220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff1b2-7f25-4fa1-8c23-d7c2957dc84e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:47.588488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:47.589141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:47.589427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:47.589702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:47.591878 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:47.664594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:47.666067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69926cd-715d-4cb1-9b7e-f099b4a19319 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:47.677838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1228/2432] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:50.549027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:50.551114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:50.631107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee49574-41a4-433a-9f52-2357a899d29d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:50.635549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:50.636327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:50.636924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:50.637444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:50.639577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:50.655433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:50.656973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbc20d-fc66-40ca-8bf1-e8fd1b0652ce service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:50.669312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1205/2433] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:51.446542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:51.447178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:51.447288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:51.448663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1229/2434] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:46:51.455411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:51.455812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:51.456210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:51.502839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] MFA Rules not processed for user `e945abf5978842a79cfaf92c479542a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:51.504111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:51.515481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:51.517714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:51.573047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1206/2435] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:51.584632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:51.586070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:51.591433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d42ac-7b50-4c00-b84f-80a461315429 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:51.596089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:51.597129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:51.597283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:51.598332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] PATH_INFO: `/v3/users/7f8df9faa4dd4eeeb724e891cdb55616` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:51.599244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8df9faa4dd4eeeb724e891cdb55616)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:51.601002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10b8c-43cd-4915-b24c-22db70d65d7d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:51.601205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:51.606331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:51.606986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:51.607310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:51.607644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:51.618009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:51.619771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:51.630906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1207/2436] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:46:51.644849 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:51.658779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8df9faa4dd4eeeb724e891cdb55616'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:51.671948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8df9faa4dd4eeeb724e891cdb55616'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:51.672683 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8df9faa4dd4eeeb724e891cdb55616'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:51.681227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8df9faa4dd4eeeb724e891cdb55616'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:51.689002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-036f1fe2-ab7a-4413-a2ef-db362a1b64ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8df9faa4dd4eeeb724e891cdb55616'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:51.694906 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1230/2437] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:46:51 2022] DELETE /identity/v3/users/7f8df9faa4dd4eeeb724e891cdb55616 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:46:51.884097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:51.895513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4bac9-eaa2-4e87-aab4-964a2e7d403f service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:51.910233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:51.960087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:51.961014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:51.961302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:51.961579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] PATH_INFO: `/v3/projects/2b043de82f1b43f78e3b92d375111d91` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:51.962787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b043de82f1b43f78e3b92d375111d91)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:51.964760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.020835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b043de82f1b43f78e3b92d375111d91', 'request_id': 'req-dbb7257b-ecd7-45d8-a517-94173b103fd0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:52.030898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b043de82f1b43f78e3b92d375111d91', 'request_id': 'req-dbb7257b-ecd7-45d8-a517-94173b103fd0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:52.038934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b043de82f1b43f78e3b92d375111d91', 'request_id': 'req-dbb7257b-ecd7-45d8-a517-94173b103fd0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:52.051658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] The token cache is being invalidate because project 2b043de82f1b43f78e3b92d375111d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:46:52.051961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:52.052596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb7257b-ecd7-45d8-a517-94173b103fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:46:52.054172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1208/2438] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:46:51 2022] DELETE /identity/v3/projects/2b043de82f1b43f78e3b92d375111d91 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:46:52.059036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.059362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.059648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.078402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:52.079682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:52.143547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1231/2439] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:52.155523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.156029 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.156517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.176399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:52.177615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:52.186378 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1209/2440] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:52.190727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.201160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1c5f2-4e16-46f6-bc47-5f7d236a5bc0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.207018 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.207717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.207998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.208275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.209006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:46:52.209312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.210938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.223243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1232/2441] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:46:52.227673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.237596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf601a-9404-4545-9f8c-2901e1b03811 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.242970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.243712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.244010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.244255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.244981 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.246914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.279107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1210/2442] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:46:52 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:46:52.285164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.300336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26895262-ac74-4c94-93cd-66cce3b4a502 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.364524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.365221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.365497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.365771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.366515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.368556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.398021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1233/2443] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:46:52 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:46:52.399353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.410580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b7e4-be1a-4a8e-a410-a403f2d65b69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.416277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.417116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.417471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.417819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.418686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.420514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.431796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1211/2444] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:46:52.437134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.449697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3d2ee-cb89-48c6-bd20-c08fa4f1f816 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.454973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.455680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.455962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.456237 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.456962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.458578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438e08b-da24-4fbf-aa32-9bd20e1303e7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.469974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1234/2445] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:46:52.474975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.487560 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c4f60-5103-4986-8d27-bd21c06a097c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.492565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.493495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.493782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.494061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] PATH_INFO: `/v3/projects/e38b2e70996d4d3ba7775bef7a60a032/users/bf4011151dc04d8cacf83c1f0930c040/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.510034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38b2e70996d4d3ba7775bef7a60a032, user_id=bf4011151dc04d8cacf83c1f0930c040, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.512596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.527798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1212/2446] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:46:52 2022] PUT /identity/v3/projects/e38b2e70996d4d3ba7775bef7a60a032/users/bf4011151dc04d8cacf83c1f0930c040/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:46:52.538294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.538587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.538887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.566792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] MFA Rules not processed for user `bf4011151dc04d8cacf83c1f0930c040`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:46:52.568052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:52.572253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:52.643153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1235/2447] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:46:52.654435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:52.656438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:46:52.710256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:46:52.723133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0559706-236c-4758-bc22-284fe06f6ebe admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:52.727877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:52.728550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:52.728839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:52.729121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:52.737818 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:52.739422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bff870-c01f-4e0a-8f27-4d47c7b1a876 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:46:52.749995 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1213/2448] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:46:59.878394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:46:59.948587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131d3f-e63d-4d65-a1d9-b8f3e71ca4c6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:46:59.954353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:46:59.955644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:46:59.956016 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:46:59.956342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] PATH_INFO: `/v3/users/bf4011151dc04d8cacf83c1f0930c040` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:46:59.957645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4011151dc04d8cacf83c1f0930c040)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:46:59.959644 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:00.010316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4011151dc04d8cacf83c1f0930c040'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.022241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4011151dc04d8cacf83c1f0930c040'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.022920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4011151dc04d8cacf83c1f0930c040'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.031648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4011151dc04d8cacf83c1f0930c040'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.040546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4011151dc04d8cacf83c1f0930c040'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.048473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1236/2449] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:46:59 2022] DELETE /identity/v3/users/bf4011151dc04d8cacf83c1f0930c040 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:00.053415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:00.053963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:00.053963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:00.072380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:00.073609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:00.136224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1214/2450] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:00.147487 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:00.149130 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:00.185121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:00.217650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338846b-e199-403b-8aa9-50eacaf10aeb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:00.222751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:00.223457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:00.223744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:00.224047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:00.234611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:00.236131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:00.246034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1237/2451] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:00.522962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:00.524495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:00.536877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed3b24-d274-42bf-9a5d-f5c58a2b771b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:00.542579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:00.543526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:00.543812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:00.544091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] PATH_INFO: `/v3/projects/e38b2e70996d4d3ba7775bef7a60a032` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:00.545513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38b2e70996d4d3ba7775bef7a60a032)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:00.547494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:00.605510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38b2e70996d4d3ba7775bef7a60a032', 'request_id': 'req-dff61b3b-1c13-4cb0-badd-4dec94f1f078'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.613978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38b2e70996d4d3ba7775bef7a60a032', 'request_id': 'req-dff61b3b-1c13-4cb0-badd-4dec94f1f078'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.621889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38b2e70996d4d3ba7775bef7a60a032', 'request_id': 'req-dff61b3b-1c13-4cb0-badd-4dec94f1f078'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.636210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] The token cache is being invalidate because project e38b2e70996d4d3ba7775bef7a60a032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:00.636581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.637528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:00.639646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1215/2452] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:00 2022] DELETE /identity/v3/projects/e38b2e70996d4d3ba7775bef7a60a032 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:47:08.320981 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:08.322685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:08.385914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7b7aa-2d86-4b8c-a2e8-bf553886a385 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:08.446287 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:08.446965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:08.447790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:08.448072 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:08.450140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:08.513696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:08.515202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5ae73-39e5-4cde-a66b-e689f444326a service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:08.525179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1238/2453] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:10.236957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:10.238701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:10.252943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff61b3b-1c13-4cb0-badd-4dec94f1f078 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:10.257206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:10.257936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:10.258231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:10.258521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:10.260567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:10.323086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:10.324582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:10.334173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1216/2454] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:18.520811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:20.330037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:22.955888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:22.956593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:22.956593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:22.979567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] MFA Rules not processed for user `fd136fcda9a04a16b258301a86bde8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:22.980958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:22.991310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1239/2455] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:23.367051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:23.369240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:23.448693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047357e-f123-4e36-8c36-52f1d28f50ee service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:23.452239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:23.452899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:23.453256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:23.453538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:23.456388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:23.466972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:23.468444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:23.478704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1217/2456] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:23.923874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:23.925585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:24.000051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720e97a-b28a-440a-9f92-8fc06af14980 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:24.005506 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:24.006167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:24.006449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:24.006725 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:24.008573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:24.024227 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:24.026633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:24.038952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1240/2457] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:25.955752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:25.957533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:26.066246 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99158203-c2fe-4fcc-9955-69811107b119 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:26.072863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:26.073642 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:26.074001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:26.074355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:26.076391 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:26.093585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:26.095569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:26.110711 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1218/2458] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:26.461837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:26.472423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79dffa9-8609-49a9-be94-9a5e2aa9bdd6 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:26.477978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:26.478647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:26.478938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:26.479252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:26.482584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:26.495877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:26.497357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:26.508029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1241/2459] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:31.771856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:31.786468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf02e6-d73c-4e18-8e9d-9a3002b46371 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:31.793859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:31.794646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:31.795035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:31.795663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:31.798010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:31.811492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:31.813195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:31.825874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1219/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:32.914047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:32.915615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:32.926639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f21e7-f92c-4b0e-9d64-0f15d775c3c1 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:32.930991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:32.931748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:32.932032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:32.932291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:32.934200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:33.006009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:33.007524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:33.017922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1242/2461] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:36.574956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:36.577168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:36.591563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa33475-9543-4166-b055-eebd5bf4cc15 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:36.597970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:36.599144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:36.599855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:36.600337 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] PATH_INFO: `/v3/users/d5e6d9fad06449c1b43254df442d224f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:36.601768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e6d9fad06449c1b43254df442d224f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:36.604766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:36.671461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e6d9fad06449c1b43254df442d224f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:36.689644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e6d9fad06449c1b43254df442d224f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:36.697276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e6d9fad06449c1b43254df442d224f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:36.698190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e6d9fad06449c1b43254df442d224f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:36.698998 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ab0cafa-6184-4f98-b056-8403318dc28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e6d9fad06449c1b43254df442d224f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:36.705703 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:36.727970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1220/2462] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:36 2022] DELETE /identity/v3/users/d5e6d9fad06449c1b43254df442d224f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:36.928332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:36.940493 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f660c-1d0e-4bcb-993e-e48831ebb2b0 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.003169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.003830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.004297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.004411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] PATH_INFO: `/v3/projects/42946f76423740028ae43dc6a49cbbda` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.005874 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42946f76423740028ae43dc6a49cbbda)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.007790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.015330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:37.061033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42946f76423740028ae43dc6a49cbbda', 'request_id': 'req-d70dd177-3df2-4518-9bc3-783f1c9c06d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:37.067384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42946f76423740028ae43dc6a49cbbda', 'request_id': 'req-d70dd177-3df2-4518-9bc3-783f1c9c06d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:37.077457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42946f76423740028ae43dc6a49cbbda', 'request_id': 'req-d70dd177-3df2-4518-9bc3-783f1c9c06d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:37.088611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] The token cache is being invalidate because project 42946f76423740028ae43dc6a49cbbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:37.088896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:37.089528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d70dd177-3df2-4518-9bc3-783f1c9c06d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:37.091122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1243/2463] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:36 2022] DELETE /identity/v3/projects/42946f76423740028ae43dc6a49cbbda => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:37.096580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.096940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.097516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.116267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:37.117455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.187256 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1221/2464] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:47:37.189002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.189258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.189568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.209168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:37.210394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.219464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1244/2465] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:37.223927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.234316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bdd20-6a5a-4d68-9552-818b81353cba None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.240072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.248128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.248424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.248700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.249437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:47:37.249744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.251381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.269961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1222/2466] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:47:37.277257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.290523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0eb08-38dd-42f1-8fcd-612ef0b345ff None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.297076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.297755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.298033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.298305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.299420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.301172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.331866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1245/2467] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:37.337886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.352107 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45b458-690d-4c9a-afe0-7cb5a69348c9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.431165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.431956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.432186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.432462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.433168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.435282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.461428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1223/2468] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/users => generated 302 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:37.462718 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.474308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e28ca-3a78-4aa3-be53-7aee09df8b46 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.478945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.479653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.479937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.480212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.480939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.482597 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.494134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1246/2469] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:37.498296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.509164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492192c0-c625-4d4c-a74d-69467d133f8a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.513943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.514624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.514917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.515232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.515970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.517775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1872-77a9-4f0a-a14d-aa5e5315978a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.535400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1224/2470] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:37.539705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.551001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28376b-935b-44db-8395-bd24dfdd944b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.555764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.556802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.557094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.557375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] PATH_INFO: `/v3/projects/f5658dc6018044a7b7ef68a000952f92/users/850d56c3de654e3bb01c6f55ff869b46/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.572976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5658dc6018044a7b7ef68a000952f92, user_id=850d56c3de654e3bb01c6f55ff869b46, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.575728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.594556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1247/2471] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:37 2022] PUT /identity/v3/projects/f5658dc6018044a7b7ef68a000952f92/users/850d56c3de654e3bb01c6f55ff869b46/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:47:37.600598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0142-a79d-4f7b-a55b-bfab4429a99e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.601100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0142-a79d-4f7b-a55b-bfab4429a99e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.601213 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0142-a79d-4f7b-a55b-bfab4429a99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.620267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-493e0142-a79d-4f7b-a55b-bfab4429a99e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:37.621485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-493e0142-a79d-4f7b-a55b-bfab4429a99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.677836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:37.691915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1225/2472] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:37.701558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.703609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.770510 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40752b-9a07-4410-95f6-6d8a3547c08e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:37.775307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:37.776027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.776433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.776743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.786984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:37.788426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:37.797492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:37.797857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:37.798142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:37.803307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1248/2473] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:37.817365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:37.818590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.896882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1226/2474] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:47:37.960717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.962193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:37.974250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:37.976426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:38.010859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:38.056521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392c455-fec2-451e-86b4-7b96e819c88a service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:38.062017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:38.062686 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.063001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.063301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.064977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359eae-17e8-4f05-93b6-5520458340f0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:38.071027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:38.071727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.072016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.072296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.074264 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:38.074465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:38.076320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb705-275e-4de3-bf2f-772a788f6f46 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:38.086560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1249/2475] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:47:38.163099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:38.164991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:38.176984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1227/2476] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:38.644591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.645141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.645429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.673382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] MFA Rules not processed for user `850d56c3de654e3bb01c6f55ff869b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:38.674859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:38.766542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1250/2477] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:38.770982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:38.784166 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b4b3-caae-4c68-aa04-1b9b06323fe3 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:38.790708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:38.791658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.792248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.792677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.793636 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:38.796873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:38.833240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1228/2478] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:38 2022] POST /identity/v3/projects => generated 405 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:38.834403 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:38.845110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01510c-f903-404a-869b-35c7c1c7f1fb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:38.903019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:38.903735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.904023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.904304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.905014 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:38.907045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:38.932935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1251/2479] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:38 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:38.934268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:38.945099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e027822-b754-4178-8309-bc6c0a0e7e4d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:38.953935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:38.954696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:38.955118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:38.955496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:38.956357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:38.958176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:38.974482 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1229/2480] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:47:38.990070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:39.004161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc055-da3c-417b-aca6-9121f5f561ad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:39.015313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:39.016814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:39.017300 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:39.019904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] PATH_INFO: `/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1/users/111bfc3088ff4717982cf2ddda5677fc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:39.040948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca9e31c7a444426a5c6d0b7c3fbeed1, user_id=111bfc3088ff4717982cf2ddda5677fc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:39.044627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:39.065373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1252/2481] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:38 2022] PUT /identity/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1/users/111bfc3088ff4717982cf2ddda5677fc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:39.066883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:39.074653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:39.083820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25be83b-c4a8-4453-87ec-1a0d84ee1ca7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:39.154230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:39.154993 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:39.155354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:39.155637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:39.156367 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:39.158122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:39.170250 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1230/2482] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:39.172304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:39.173985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:39.187326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848adad7-b014-46c7-8cff-1a672d3e6661 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:39.194950 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:39.195909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:39.196207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:39.196461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] PATH_INFO: `/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1/users/111bfc3088ff4717982cf2ddda5677fc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:39.199051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca9e31c7a444426a5c6d0b7c3fbeed1, user_id=111bfc3088ff4717982cf2ddda5677fc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:39.202034 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbdded-d2f3-47a1-aea3-80f3bb70d260 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:39.210389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:39.218369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1231/2483] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:39 2022] PUT /identity/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1/users/111bfc3088ff4717982cf2ddda5677fc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:47:39.317890 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06feafd-bd60-46c8-81d7-2eb27fc6a0ad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:39.396864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:39.397723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:39.398043 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:39.398373 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:39.400360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:39.415095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:39.416743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:39.439104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1253/2484] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:39.974499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:39.974991 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:39.975138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:40.009374 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] MFA Rules not processed for user `111bfc3088ff4717982cf2ddda5677fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:40.010696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:40.095816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1232/2485] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:41.624490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:41.727217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f123b-46df-47b4-8a8a-32f62d0c9562 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:41.735676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:41.736939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:41.737409 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:41.737830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] PATH_INFO: `/v3/users/850d56c3de654e3bb01c6f55ff869b46` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:41.739522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=850d56c3de654e3bb01c6f55ff869b46)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:41.742248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:41.793327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:41.796049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850d56c3de654e3bb01c6f55ff869b46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:41.813473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850d56c3de654e3bb01c6f55ff869b46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:41.814230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850d56c3de654e3bb01c6f55ff869b46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:41.823181 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850d56c3de654e3bb01c6f55ff869b46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:41.830893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850d56c3de654e3bb01c6f55ff869b46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:41.837269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1254/2486] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:41 2022] DELETE /identity/v3/users/850d56c3de654e3bb01c6f55ff869b46 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:42.036323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:42.059829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ebe96-2ce0-47ca-a862-72ad8e22216a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:42.152522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:42.153744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:42.154171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:42.154590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] PATH_INFO: `/v3/projects/f5658dc6018044a7b7ef68a000952f92` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:42.156210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5658dc6018044a7b7ef68a000952f92)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:42.160175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:42.181893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:42.241785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5658dc6018044a7b7ef68a000952f92', 'request_id': 'req-bb958c96-a7c6-4460-9a95-46f956383624'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.255856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5658dc6018044a7b7ef68a000952f92', 'request_id': 'req-bb958c96-a7c6-4460-9a95-46f956383624'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.266485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5658dc6018044a7b7ef68a000952f92', 'request_id': 'req-bb958c96-a7c6-4460-9a95-46f956383624'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.282361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] The token cache is being invalidate because project f5658dc6018044a7b7ef68a000952f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:42.283049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.283826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.285637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1233/2487] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:42 2022] DELETE /identity/v3/projects/f5658dc6018044a7b7ef68a000952f92 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:42.289738 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:42.291344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:42.310821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1dff2-b788-48cd-87a2-098c7b38bc8e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:42.417368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:42.419968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:42.422210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:42.422633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] PATH_INFO: `/v3/users/111bfc3088ff4717982cf2ddda5677fc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:42.424419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111bfc3088ff4717982cf2ddda5677fc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:42.427118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:42.509292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111bfc3088ff4717982cf2ddda5677fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.524574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111bfc3088ff4717982cf2ddda5677fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.525625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111bfc3088ff4717982cf2ddda5677fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.536702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111bfc3088ff4717982cf2ddda5677fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.546249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-831dac3c-8b49-4495-ae12-781603000ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111bfc3088ff4717982cf2ddda5677fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:42.554325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1255/2488] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:42 2022] DELETE /identity/v3/users/111bfc3088ff4717982cf2ddda5677fc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:43.046442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.049065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:43.064619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb958c96-a7c6-4460-9a95-46f956383624 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.159977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.161052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.161473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.161817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] PATH_INFO: `/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.163260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca9e31c7a444426a5c6d0b7c3fbeed1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.165318 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.224909 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca9e31c7a444426a5c6d0b7c3fbeed1', 'request_id': 'req-2c142c83-78b1-4a4b-840b-5f24107a638b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:43.236792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca9e31c7a444426a5c6d0b7c3fbeed1', 'request_id': 'req-2c142c83-78b1-4a4b-840b-5f24107a638b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:43.245052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca9e31c7a444426a5c6d0b7c3fbeed1', 'request_id': 'req-2c142c83-78b1-4a4b-840b-5f24107a638b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:43.258895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] The token cache is being invalidate because project 5ca9e31c7a444426a5c6d0b7c3fbeed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:43.259178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:43.259910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c142c83-78b1-4a4b-840b-5f24107a638b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:43.263227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1234/2489] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:43 2022] DELETE /identity/v3/projects/5ca9e31c7a444426a5c6d0b7c3fbeed1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:43.267602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.268318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.268987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.293012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:43.294334 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:43.342186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:43.375163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1256/2490] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:43.382230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.382461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.382765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.412633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:43.415205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:43.433438 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1235/2491] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:43.434215 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.447720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f1094-24b0-475b-942d-d533c4d5019d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.459665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.471331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.471715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.472012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.472988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:47:43.473307 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.475202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.494953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1257/2492] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:47:43.496510 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.510601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c3c48-b365-4504-8bcf-ba32d200011c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.516805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.521017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.522435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.522435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.522917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.525574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.576292 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1236/2493] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:43.580536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.594734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bdca4-37c9-421e-9d8f-9315c67285d3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.665040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.666031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.666523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.667133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.668209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.670402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.699007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1258/2494] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:43.703639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.709945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:43.717450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663698-27c1-4766-97d9-908176a6028f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.724921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.725748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.726030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.726325 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.727152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.728804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.742441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1237/2495] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:43.746693 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.757467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dc8cd-f045-4201-8eff-6708bd607716 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.762480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.764898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.765276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.765675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.767177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.769191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2154ef-5d19-45d5-8171-9d0af11e69ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.782282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1259/2496] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:47:43.786708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:43.799117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787526a1-ed9d-4965-9457-7abe4a2868bb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:43.804156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:43.805074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.805361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.805639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] PATH_INFO: `/v3/projects/fafae01433834fb28d2858f79440c73c/users/18ca898e3aaf4dd99d0b591117dbf944/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.825511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fafae01433834fb28d2858f79440c73c, user_id=18ca898e3aaf4dd99d0b591117dbf944, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:43.830948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:43.851612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1238/2497] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:43 2022] PUT /identity/v3/projects/fafae01433834fb28d2858f79440c73c/users/18ca898e3aaf4dd99d0b591117dbf944/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:43.862788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:43.864159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:43.865411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:43.902734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] MFA Rules not processed for user `18ca898e3aaf4dd99d0b591117dbf944`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:43.904269 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:43.998879 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.002925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1260/2498] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:44.078720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583083-8548-42d0-a752-4d1d7653bd49 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.084087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.084773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.085069 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.085349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.086097 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.088061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.119156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1239/2499] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:43 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:44.121983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.134430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e28d0-8885-47f8-8b84-119d7d391868 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.177210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:44.211882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.212623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.212907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.213182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.213889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.215869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.249836 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1261/2500] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:44 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:44.251160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.264249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea484a0-3f2c-434f-8a60-9d8f69cd1a29 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.269944 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.270778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.271067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.271369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.272099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.273740 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.290629 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1240/2501] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:44.292002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.303521 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50254c16-c600-4146-87a6-0e08298c7652 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.308635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.309566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.309817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.310096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] PATH_INFO: `/v3/projects/647e201f2ed84cac9a41552923c111f1/users/dcc8df5e941645e6af8fe0d2fa2b9fb8/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.331292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647e201f2ed84cac9a41552923c111f1, user_id=dcc8df5e941645e6af8fe0d2fa2b9fb8, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.335032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.357206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1262/2502] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:44 2022] PUT /identity/v3/projects/647e201f2ed84cac9a41552923c111f1/users/dcc8df5e941645e6af8fe0d2fa2b9fb8/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:44.358443 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.371994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236fdea-12ea-43be-9ad6-071b6b775ec6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.441326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.442017 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.442298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.442576 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.443344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.445505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53384c17-9ee2-47aa-af5b-8829e19f9050 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.460645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1241/2503] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:47:44.464060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.475353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c71c-8d37-44a7-ae94-bd6004f2ac62 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.480034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.480944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.481232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.481511 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] PATH_INFO: `/v3/projects/647e201f2ed84cac9a41552923c111f1/users/dcc8df5e941645e6af8fe0d2fa2b9fb8/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.483539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647e201f2ed84cac9a41552923c111f1, user_id=dcc8df5e941645e6af8fe0d2fa2b9fb8, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.486473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.499887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1263/2504] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:44 2022] PUT /identity/v3/projects/647e201f2ed84cac9a41552923c111f1/users/dcc8df5e941645e6af8fe0d2fa2b9fb8/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:44.516305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.516654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.516962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.554624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] MFA Rules not processed for user `dcc8df5e941645e6af8fe0d2fa2b9fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:44.555941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:44.591872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:44.651454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1242/2505] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:47:44.662788 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.664536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:44.731548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3281e10-9dab-4b44-8e80-c9c5d0fc4da3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.736376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.737617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.737617 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.737864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.755178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.756775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.770682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1264/2506] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:44.856658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:44.866614 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac016-79d7-40bc-9d24-ab3a165ff38c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:44.870409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:44.871126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:44.871400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:44.871678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:44.937410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:44.938921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:44.948434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1243/2507] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:47.767697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:47.767697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:47.851235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e370c-7605-494c-8a7d-dd525eec7ae9 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:47.857945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:47.859540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:47.860159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:47.860722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] PATH_INFO: `/v3/users/1636311c73b9461c9a64db18e8fdf0a3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:47.862895 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1636311c73b9461c9a64db18e8fdf0a3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:47.866343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:47.922498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:47.944817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1636311c73b9461c9a64db18e8fdf0a3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:47.964776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1636311c73b9461c9a64db18e8fdf0a3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:47.965922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1636311c73b9461c9a64db18e8fdf0a3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:47.980919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1636311c73b9461c9a64db18e8fdf0a3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:47.989898 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1636311c73b9461c9a64db18e8fdf0a3'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:47.997549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1265/2508] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:47 2022] DELETE /identity/v3/users/1636311c73b9461c9a64db18e8fdf0a3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:48.229267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:48.254804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63560b70-28a3-4755-bd13-6e72e54122a5 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:48.330898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:48.331823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:48.332109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:48.332513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] PATH_INFO: `/v3/projects/a0aff51028ec477ebbc12e078174977a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:48.333722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0aff51028ec477ebbc12e078174977a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:48.335812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:48.382124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0aff51028ec477ebbc12e078174977a', 'request_id': 'req-f4e24616-c30a-485a-9a55-1f088b4ed8e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.390048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0aff51028ec477ebbc12e078174977a', 'request_id': 'req-f4e24616-c30a-485a-9a55-1f088b4ed8e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.395224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:48.396889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0aff51028ec477ebbc12e078174977a', 'request_id': 'req-f4e24616-c30a-485a-9a55-1f088b4ed8e0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.410919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] The token cache is being invalidate because project a0aff51028ec477ebbc12e078174977a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:48.411248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.411983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.413651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1244/2509] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:48 2022] DELETE /identity/v3/projects/a0aff51028ec477ebbc12e078174977a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:48.417888 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:48.419479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:48.433941 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f05df-edbb-44b2-8577-17333ebd166d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:48.491622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:48.492428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:48.492700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:48.492987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] PATH_INFO: `/v3/users/a1d741ae3bf1446eb81d03d93b673830` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:48.494167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d741ae3bf1446eb81d03d93b673830)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:48.496150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:48.542528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d741ae3bf1446eb81d03d93b673830'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.554104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d741ae3bf1446eb81d03d93b673830'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.554769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d741ae3bf1446eb81d03d93b673830'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.562775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d741ae3bf1446eb81d03d93b673830'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.570630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d741ae3bf1446eb81d03d93b673830'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.576692 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1266/2510] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:48 2022] DELETE /identity/v3/users/a1d741ae3bf1446eb81d03d93b673830 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:48.757005 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:48.759693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:48.780289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24616-c30a-485a-9a55-1f088b4ed8e0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:48.846650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:48.847611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:48.847875 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:48.848169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] PATH_INFO: `/v3/projects/98bea12852fc45879f303aa81b795485` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:48.849526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98bea12852fc45879f303aa81b795485)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:48.851539 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:48.901516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98bea12852fc45879f303aa81b795485', 'request_id': 'req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.910820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98bea12852fc45879f303aa81b795485', 'request_id': 'req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.918579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98bea12852fc45879f303aa81b795485', 'request_id': 'req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.931243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] The token cache is being invalidate because project 98bea12852fc45879f303aa81b795485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:48.931565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.932249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-56b708ac-0b02-44ba-9f27-ad1d8f53fc74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:48.933834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1245/2511] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:48 2022] DELETE /identity/v3/projects/98bea12852fc45879f303aa81b795485 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:54.392105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:54.394101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:54.408077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fb9ed-3414-4b8b-84c2-35e0d1b1fef0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:54.441330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:54.484296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:54.485263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:54.485625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:54.485916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] PATH_INFO: `/v3/users/18ca898e3aaf4dd99d0b591117dbf944` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:54.487308 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ca898e3aaf4dd99d0b591117dbf944)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:54.489731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:54.546081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ca898e3aaf4dd99d0b591117dbf944'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:54.559637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ca898e3aaf4dd99d0b591117dbf944'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:54.560447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ca898e3aaf4dd99d0b591117dbf944'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:54.569628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ca898e3aaf4dd99d0b591117dbf944'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:54.577232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-db6bc388-c6ce-4fe5-84df-f16ebee2d148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ca898e3aaf4dd99d0b591117dbf944'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:54.584923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1267/2512] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:54 2022] DELETE /identity/v3/users/18ca898e3aaf4dd99d0b591117dbf944 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:54.590695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:54.591662 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:54.592411 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:54.610216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:54.611473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:54.646665 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:54.646963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:54.647285 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:54.664451 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:54.665670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:54.676806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1246/2513] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:54.683408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:54.684775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:54.714620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:54.723976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1268/2514] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:47:54.737951 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:54.739728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:54.764204 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6afe9d-1bcd-4f28-9633-d54054646b74 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:54.768092 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:54.768776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:54.769048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:54.769327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:54.780535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:54.782645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:54.795150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1247/2515] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:47:54.803651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:54.821301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e723808-8726-4717-b315-ca6e3dff7c18 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:54.826830 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:54.827542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:54.827832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:54.828109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:54.837306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:54.838942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:54.847948 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1269/2516] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:47:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 09:47:55.113767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:55.116072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:55.143888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc30aae-bf75-42ae-b77b-ef5777192dcd service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:55.151778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:55.153046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:55.153679 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:55.154173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] PATH_INFO: `/v3/projects/fafae01433834fb28d2858f79440c73c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:55.156013 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fafae01433834fb28d2858f79440c73c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:55.158317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:55.263787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fafae01433834fb28d2858f79440c73c', 'request_id': 'req-71087af1-fce8-4de7-8579-6586fe6fa680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.277396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fafae01433834fb28d2858f79440c73c', 'request_id': 'req-71087af1-fce8-4de7-8579-6586fe6fa680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.284311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fafae01433834fb28d2858f79440c73c', 'request_id': 'req-71087af1-fce8-4de7-8579-6586fe6fa680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.297643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] The token cache is being invalidate because project fafae01433834fb28d2858f79440c73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:55.297945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.301699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.307018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1248/2517] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:55 2022] DELETE /identity/v3/projects/fafae01433834fb28d2858f79440c73c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:47:55.307585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:55.329206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d95b-9d18-4673-914b-9d7a8a0dfca4 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:55.419770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:55.420786 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:55.421372 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:55.422030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] PATH_INFO: `/v3/users/dcc8df5e941645e6af8fe0d2fa2b9fb8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:55.424883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc8df5e941645e6af8fe0d2fa2b9fb8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:55.427463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:55.494217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc8df5e941645e6af8fe0d2fa2b9fb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.509061 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc8df5e941645e6af8fe0d2fa2b9fb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.510460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc8df5e941645e6af8fe0d2fa2b9fb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.524007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc8df5e941645e6af8fe0d2fa2b9fb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.534221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-85806b53-cb15-40c2-9eae-bc2dd7010d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc8df5e941645e6af8fe0d2fa2b9fb8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:55.544468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1270/2518] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:55 2022] DELETE /identity/v3/users/dcc8df5e941645e6af8fe0d2fa2b9fb8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:55.892973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:55.895403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:55.920231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71087af1-fce8-4de7-8579-6586fe6fa680 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.002269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.003221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.003504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.003778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] PATH_INFO: `/v3/projects/647e201f2ed84cac9a41552923c111f1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.004989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=647e201f2ed84cac9a41552923c111f1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.007108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.065861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647e201f2ed84cac9a41552923c111f1', 'request_id': 'req-093910e8-d916-473a-be3f-1b06f143d3dd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:56.067906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:56.075878 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647e201f2ed84cac9a41552923c111f1', 'request_id': 'req-093910e8-d916-473a-be3f-1b06f143d3dd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:56.085550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647e201f2ed84cac9a41552923c111f1', 'request_id': 'req-093910e8-d916-473a-be3f-1b06f143d3dd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:56.099185 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] The token cache is being invalidate because project 647e201f2ed84cac9a41552923c111f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:56.099483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:56.100267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-093910e8-d916-473a-be3f-1b06f143d3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:56.103032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1249/2519] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:55 2022] DELETE /identity/v3/projects/647e201f2ed84cac9a41552923c111f1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:56.108032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.108396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.108748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.131942 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:56.133400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:56.210315 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1271/2520] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:56.218924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.219317 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.219960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.241976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:56.243220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:56.252405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1250/2521] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:56.256329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.261914 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:56.271552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae0316-1b60-4785-8817-a4428a49f4c3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.277501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.278556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.279603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.279603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.280425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:47:56.280899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.285296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.304525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1272/2522] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:47:56.305468 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.317233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c959a-34d6-4a7c-87e7-dd453ebccefa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.322484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.323311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.323603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.323868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.324573 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.326566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.358106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1251/2523] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:47:56.362435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.374947 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24313-8279-4855-8e1d-0f995ef1de0d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.446964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.448349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.448349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.448595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.449226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.451426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.493529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1273/2524] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:56.497731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.510810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534c27-1adc-4e1a-b38c-004dbb69bf69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.516708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.518019 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.518523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.519056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.520373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.522243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.537252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1252/2525] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:56.541217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.552420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907b054-082a-4447-87d4-fe369ebb31dd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.557402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.558126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.558416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.558713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.559492 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.561128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa0870-3987-4faf-bf20-b9b76102095b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.576095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1274/2526] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:56.579413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.590144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742260a-1efa-425d-906a-206d36820086 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.595888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.599245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.599245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.599245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] PATH_INFO: `/v3/projects/4f3b3798c30144aaae21e253d773e63e/users/e9d5b546c7ae4c3e8f41c6441eb6808c/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.616371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3b3798c30144aaae21e253d773e63e, user_id=e9d5b546c7ae4c3e8f41c6441eb6808c, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.618030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.635327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1253/2527] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:56 2022] PUT /identity/v3/projects/4f3b3798c30144aaae21e253d773e63e/users/e9d5b546c7ae4c3e8f41c6441eb6808c/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:56.645982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.646371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.646724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.677057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] MFA Rules not processed for user `e9d5b546c7ae4c3e8f41c6441eb6808c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:56.678616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:56.764089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1275/2528] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:56.765113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.843020 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094579a8-38e2-40ee-aa13-ab7b6462adc5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.848912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.849600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.849880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.850153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.850857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.852824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.881447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1254/2529] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:56.885549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:56.898396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0142af9-299b-499b-b8cd-b6ba68c5c97f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:56.954110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:56.962939 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:56.963791 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:56.964142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:56.964488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:56.965257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:56.967541 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:56.994381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1276/2530] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:47:56 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:56.998672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.011592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532d2ac-9ea7-43a6-a6b7-c114628b7991 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.016540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.017360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.017744 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.018754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.019009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.020766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:57.035939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1255/2531] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:57.040590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.054966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7813e-57c7-426e-a76b-c36bc868a7db admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.061810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.062845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.063165 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.065330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] PATH_INFO: `/v3/projects/da74eab7653a40dd9359d444c590e702/users/f0e51c709dc843f2b7e8f10b72264c1b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.085197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da74eab7653a40dd9359d444c590e702, user_id=f0e51c709dc843f2b7e8f10b72264c1b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.088410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:57.120662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1277/2532] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:57 2022] PUT /identity/v3/projects/da74eab7653a40dd9359d444c590e702/users/f0e51c709dc843f2b7e8f10b72264c1b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:57.122491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.147000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef4820-8325-48af-8397-4501f984eea1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.217082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:57.232242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.232978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.233619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.234526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.235529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.237788 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054ccf6-15a8-464f-b007-3ed6238a8fed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:57.251015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1256/2533] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:47:57.255590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.271668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d246017-5ebd-4d78-9a52-6d000c71a676 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.284049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.284977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.285258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.285539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] PATH_INFO: `/v3/projects/da74eab7653a40dd9359d444c590e702/users/f0e51c709dc843f2b7e8f10b72264c1b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.287587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da74eab7653a40dd9359d444c590e702, user_id=f0e51c709dc843f2b7e8f10b72264c1b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.290268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:57.311608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1278/2534] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:47:57 2022] PUT /identity/v3/projects/da74eab7653a40dd9359d444c590e702/users/f0e51c709dc843f2b7e8f10b72264c1b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:57.313618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.313927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.314187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.349061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] MFA Rules not processed for user `f0e51c709dc843f2b7e8f10b72264c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:57.350332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:57.437294 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1257/2535] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:57.450490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.452410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:57.528162 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04e8e-2aba-4337-9cca-3a834f61b68b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.532455 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.533974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.534229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.534648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.548033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.552539 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:57.564298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1279/2536] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:57.977506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:57.990720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efee87e-dcfc-4155-bf90-698f5f1d1f1a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:57.995398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e51c709dc843f2b7e8f10b72264c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da74eab7653a40dd9359d444c590e702', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:57.996075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:57.998205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:57.998205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:57.998205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:47:57.998205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:57.999693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:58.013664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1258/2537] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:47:58.017386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:58.027752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b3ed2-5bdb-4500-918c-b598fc9775fd service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:58.037293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e51c709dc843f2b7e8f10b72264c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da74eab7653a40dd9359d444c590e702', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:58.038163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:58.038524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:58.038873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:58.039700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:58.041708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:58.078763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1280/2538] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:47:58 2022] POST /identity/v3/projects => generated 353 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:47:58.079989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:58.094335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2db75-755e-4a00-8898-efc1d4b50428 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:58.163582 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e51c709dc843f2b7e8f10b72264c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da74eab7653a40dd9359d444c590e702', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:58.164270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:58.164555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:58.164837 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:58.165700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:47:58.166064 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:58.167751 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:58.185218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1259/2539] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:47:58.228995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:58.243682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe4ae-78c2-4328-b6d4-2ad8eaed204c tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:58.248854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e51c709dc843f2b7e8f10b72264c1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da74eab7653a40dd9359d444c590e702', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:58.249817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:58.250186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:58.250537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] PATH_INFO: `/v3/projects/9ec2953ef8ae4abeb60a8db5fd233b09` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:58.252801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9ec2953ef8ae4abeb60a8db5fd233b09)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:58.254872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:58.309640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec2953ef8ae4abeb60a8db5fd233b09', 'request_id': 'req-d7acdafb-ddce-430f-8eca-b7ce2b466444'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.318570 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec2953ef8ae4abeb60a8db5fd233b09', 'request_id': 'req-d7acdafb-ddce-430f-8eca-b7ce2b466444'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.330020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec2953ef8ae4abeb60a8db5fd233b09', 'request_id': 'req-d7acdafb-ddce-430f-8eca-b7ce2b466444'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.343599 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] The token cache is being invalidate because project 9ec2953ef8ae4abeb60a8db5fd233b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:58.343925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.345070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7acdafb-ddce-430f-8eca-b7ce2b466444 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.347037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1281/2540] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:58 2022] DELETE /identity/v3/projects/9ec2953ef8ae4abeb60a8db5fd233b09 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:58.566429 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:58.568268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:58.636442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:58.653475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e44364-a42c-4a19-99b1-5f4d7579ef38 tempest-VolumeQuotaClassesTest-1067555624 tempest-VolumeQuotaClassesTest-1067555624-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:58.715987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:58.716916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:58.717266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:58.717572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] PATH_INFO: `/v3/users/e9d5b546c7ae4c3e8f41c6441eb6808c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:58.718834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d5b546c7ae4c3e8f41c6441eb6808c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:58.721956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:58.791874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d5b546c7ae4c3e8f41c6441eb6808c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.799922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d5b546c7ae4c3e8f41c6441eb6808c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.812528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d5b546c7ae4c3e8f41c6441eb6808c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.813459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d5b546c7ae4c3e8f41c6441eb6808c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.815517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d5b546c7ae4c3e8f41c6441eb6808c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:58.843988 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1260/2541] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:58 2022] DELETE /identity/v3/users/e9d5b546c7ae4c3e8f41c6441eb6808c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:47:58.849342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:58.849903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:58.849903 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:58.888512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:47:58.889745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:58.902697 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:58.977891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1282/2542] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:47:58.989270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:58.990856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:59.053103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1267ea-99f1-42bf-b89f-fe588ead9ed5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:59.059018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:59.059825 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:59.060209 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:59.061193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:59.075448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:59.077153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:59.089836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1261/2543] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:47:59.420630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:59.422416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:47:59.458811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae24947-3691-4e25-bd75-a4701812da4c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:59.463893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:59.464784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:59.465064 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:59.465436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] PATH_INFO: `/v3/projects/4f3b3798c30144aaae21e253d773e63e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:59.466626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3b3798c30144aaae21e253d773e63e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:59.468607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:59.525652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3b3798c30144aaae21e253d773e63e', 'request_id': 'req-47287947-7e6f-41e5-91a5-8fce83cd82f1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.532642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3b3798c30144aaae21e253d773e63e', 'request_id': 'req-47287947-7e6f-41e5-91a5-8fce83cd82f1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.541740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3b3798c30144aaae21e253d773e63e', 'request_id': 'req-47287947-7e6f-41e5-91a5-8fce83cd82f1'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.554147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] The token cache is being invalidate because project 4f3b3798c30144aaae21e253d773e63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:47:59.554434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.555216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.557113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1283/2544] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:47:59 2022] DELETE /identity/v3/projects/4f3b3798c30144aaae21e253d773e63e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:47:59.567504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:47:59.579086 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4e60-c826-44bd-a22f-84746bbe3a7d service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:47:59.604903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:47:59.660503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:47:59.661585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:47:59.662061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:47:59.662399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] PATH_INFO: `/v3/users/f0e51c709dc843f2b7e8f10b72264c1b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:47:59.663724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e51c709dc843f2b7e8f10b72264c1b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:47:59.665721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:47:59.729801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e51c709dc843f2b7e8f10b72264c1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.740437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e51c709dc843f2b7e8f10b72264c1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.753783 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e51c709dc843f2b7e8f10b72264c1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.754550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e51c709dc843f2b7e8f10b72264c1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.755270 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd04d7fc-e9db-4eb9-b115-254bb70c34a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e51c709dc843f2b7e8f10b72264c1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:47:59.782104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1262/2545] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:47:59 2022] DELETE /identity/v3/users/f0e51c709dc843f2b7e8f10b72264c1b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:00.193728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.195787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:00.216170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287947-7e6f-41e5-91a5-8fce83cd82f1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.290741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.291668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.291955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.292225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] PATH_INFO: `/v3/projects/da74eab7653a40dd9359d444c590e702` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.293404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=da74eab7653a40dd9359d444c590e702)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.295413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.348090 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:00.357946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da74eab7653a40dd9359d444c590e702', 'request_id': 'req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:00.365876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da74eab7653a40dd9359d444c590e702', 'request_id': 'req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:00.375035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da74eab7653a40dd9359d444c590e702', 'request_id': 'req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:00.387555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] The token cache is being invalidate because project da74eab7653a40dd9359d444c590e702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:00.387945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:00.388635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7cd40ea-1dbf-4c5a-a2f3-5adec62359ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:00.390210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1284/2546] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:00 2022] DELETE /identity/v3/projects/da74eab7653a40dd9359d444c590e702 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:00.395348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.395874 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.395983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.419139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:00.420386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:00.495607 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1263/2547] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:00.504075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.504457 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.504807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.525379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:00.526730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:00.535484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1285/2548] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:00.540030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.550171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f92d-0b22-45af-bc82-fb92ec0cfdfc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.554939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.555653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.555934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.556206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.557218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:00.557527 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.559190 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.572257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1264/2549] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:48:00.576332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.586343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc56d9-9609-49c8-8ad9-592625faef1f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.594714 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.596036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.596533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.596971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.598263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.600579 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.650509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1286/2550] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:00.654821 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.665302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b60ee-d29e-483e-a63c-d21d9906e014 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.691124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:00.739230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.741136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.741476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.742351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.743102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.747144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.780512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1265/2551] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:00 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:00.784958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.796329 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcdc41-3f3c-4680-a200-f0b48e619a5f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.801477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.802268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.802654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.803026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.804055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.806569 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.820463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1287/2552] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:00.824172 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.836236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd076b-cbcf-4898-a505-4f57245aad14 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.841410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.841690 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.841976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.842251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.842989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.844999 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7394c-615c-4e77-8f08-5631b888767a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.856473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1266/2553] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:48:00.860516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:00.872013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd7244-eb84-40e6-b086-7690e4665e42 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:00.876824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:00.877881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.878172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.878432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] PATH_INFO: `/v3/projects/e7753d5879514268b8ad39e1b1b6dd3b/users/786721d4cca643d2839b3500dd3849d9/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.894021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7753d5879514268b8ad39e1b1b6dd3b, user_id=786721d4cca643d2839b3500dd3849d9, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:00.896831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:00.913304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1288/2554] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:00 2022] PUT /identity/v3/projects/e7753d5879514268b8ad39e1b1b6dd3b/users/786721d4cca643d2839b3500dd3849d9/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:48:00.918415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2f3b-329a-41c3-8e47-ac45ac04f390 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:00.918691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2f3b-329a-41c3-8e47-ac45ac04f390 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:00.918971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2f3b-329a-41c3-8e47-ac45ac04f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:00.940519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4dff2f3b-329a-41c3-8e47-ac45ac04f390 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:00.941887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4dff2f3b-329a-41c3-8e47-ac45ac04f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:01.006420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1267/2555] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:01.017094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:01.019641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:01.084295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0f41-1615-4c30-a820-aaa9eac614ac admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:01.088242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:01.088907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:01.089306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:01.089586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:01.100065 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:01.101522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:01.112355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1289/2556] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:01.935769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:01.936303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:01.936400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:01.965776 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] MFA Rules not processed for user `786721d4cca643d2839b3500dd3849d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:01.967186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:02.070912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1268/2557] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:02.075842 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.093304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47735a-a509-4f97-99ef-7628b0785e39 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.100754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.101689 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.102619 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.103755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.104606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.109111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.146500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1290/2558] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:02 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:02.150863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.168109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591acc2-5548-4a43-84ca-94872e523129 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.233651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:02.269848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.270548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.270823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.271341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.272040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.274401 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.304093 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1269/2559] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:02 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:48:02.307845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.324394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb150b5b-983b-4bd7-9142-e628a9bf2c8b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.331856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.332807 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.333097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.333371 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.334102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.335797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.350640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1291/2560] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:02.352891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.364551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcfdc3-465f-4e98-bb58-c8d6f57ff3ac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.370903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.370903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.370903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.370903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] PATH_INFO: `/v3/projects/18cf7994731a433daa60dd220c293a3c/users/b885586eb931442c968a7b3c1e6826f0/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.393111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf7994731a433daa60dd220c293a3c, user_id=b885586eb931442c968a7b3c1e6826f0, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.395870 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.426131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.427240 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1270/2561] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:02 2022] PUT /identity/v3/projects/18cf7994731a433daa60dd220c293a3c/users/b885586eb931442c968a7b3c1e6826f0/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:02.442207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b243-a1c9-46fb-895f-b6011d70dfd8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.478713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:02.517805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.518729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.519127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.519498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.520342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.522353 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910ce81-c3e4-4480-bb8a-6cd683ec5ae5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.536995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1292/2562] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:02.543483 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:02.556814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f07a5f-eb8f-4a40-9c7e-60708b4fc625 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:02.564802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:02.566004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:02.566445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:02.566881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] PATH_INFO: `/v3/projects/18cf7994731a433daa60dd220c293a3c/users/b885586eb931442c968a7b3c1e6826f0/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:02.569328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf7994731a433daa60dd220c293a3c, user_id=b885586eb931442c968a7b3c1e6826f0, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:02.572263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:02.591037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1271/2563] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:02 2022] PUT /identity/v3/projects/18cf7994731a433daa60dd220c293a3c/users/b885586eb931442c968a7b3c1e6826f0/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:03.419038 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:03.427639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:03.427639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:03.457185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] MFA Rules not processed for user `b885586eb931442c968a7b3c1e6826f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:03.459094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:03.573212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1293/2564] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:03.586955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:03.588732 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:03.667652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641138a-02a2-4e02-8427-2434397a1f5f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:03.676842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:03.677618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:03.677962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:03.678301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:03.755386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:03.757009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:03.773568 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1272/2565] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:05.231495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:05.233467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:05.246668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225973f-ede4-407f-8311-5023b32f67d5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:05.251203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:05.251814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:05.252096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:05.252386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:05.262565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:05.264080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97e4c9-7e8e-4e4b-9063-b6876baf72da service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:05.274303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1294/2566] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:08.425328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:08.486040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:08.499028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e796612-fdc1-45fb-a12b-8a59e17e7953 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:08.502947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:08.504179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.504465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.504741 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.506736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:08.517975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:08.519986 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d9a11-3954-4b79-ac31-2dff53ca01a9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:08.528351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1273/2567] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:08.640499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.641051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.641179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.642764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1295/2568] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:48:08.649566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b162c-4ba0-4aba-aff1-94a513ea34e1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.649931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b162c-4ba0-4aba-aff1-94a513ea34e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.650241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b162c-4ba0-4aba-aff1-94a513ea34e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.671578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa6b162c-4ba0-4aba-aff1-94a513ea34e1 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:08.672796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa6b162c-4ba0-4aba-aff1-94a513ea34e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:08.749475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1274/2569] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:08.764071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:08.766236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:08.826119 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e077-5655-4a68-b664-ce8a4610c658 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:08.830431 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:08.831105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.831382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.831664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.842401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:08.843884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fec67-2fe2-461d-8b13-f97bfa3a1d80 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:08.855134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1296/2570] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:08.927819 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.928112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.928390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.929787 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1275/2571] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:48:08.936964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4642b-df82-425f-a115-698e40a73eef None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:08.937252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4642b-df82-425f-a115-698e40a73eef None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:08.937533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4642b-df82-425f-a115-698e40a73eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:08.958268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-61b4642b-df82-425f-a115-698e40a73eef None None] MFA Rules not processed for user `2381343893e74c0a919ff4d0d59b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:08.959503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-61b4642b-df82-425f-a115-698e40a73eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:08.967559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1297/2572] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:08.980001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:08.981916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:08.996672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bba5a7-efdf-4426-931e-68925c1a3b76 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:09.000474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:09.001015 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:09.001298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:09.001572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:09.008772 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:09.010216 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29025c12-fec9-426c-b149-330fe2df478d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:09.018400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1276/2573] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:09.103937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8eee0-ab52-4913-bb2b-31bb01a05355 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:09.104288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8eee0-ab52-4913-bb2b-31bb01a05355 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:09.104564 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8eee0-ab52-4913-bb2b-31bb01a05355 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:09.106232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1298/2574] 10.222.0.65 () {54 vars in 921 bytes} [Fri Jun 24 09:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 09:48:09.111889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f8bf-4528-4d8c-84fb-323195c7a783 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:09.112463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f8bf-4528-4d8c-84fb-323195c7a783 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:09.113043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9f8bf-4528-4d8c-84fb-323195c7a783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:09.132606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eda9f8bf-4528-4d8c-84fb-323195c7a783 None None] MFA Rules not processed for user `15d802693dbc490c890d922d838f856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:09.133847 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eda9f8bf-4528-4d8c-84fb-323195c7a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:09.143485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1277/2575] 10.222.0.65 () {58 vars in 1018 bytes} [Fri Jun 24 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:09.155383 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8eee0-ab52-4913-bb2b-31bb01a05355 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:09.170075 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8eee0-ab52-4913-bb2b-31bb01a05355 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:09.174809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:09.176763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:09.176763 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:09.176962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:09.190314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:09.192381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:09.204245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1299/2576] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:10.882594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:10.883137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:10.883249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:10.903476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:10.904655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:10.964801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1278/2577] 10.222.0.65 () {58 vars in 1009 bytes} [Fri Jun 24 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:10.978634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:10.980395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:11.013922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:11.065676 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8c192-e90a-4516-a6eb-be984358b044 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:11.071218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:11.071921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:11.072206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:11.072478 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:11.081271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:11.083199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:11.091633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1300/2578] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:48:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:48:18.773973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:18.775456 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:18.848128 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:18.869022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:18.878932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44db3a-5452-474c-aa5a-0337a479b38f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:18.886047 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:18.886820 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:18.887156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:18.887424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:18.889812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:18.906764 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:18.909073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:18.941156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1279/2579] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:18.959925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da729087-6570-4853-b4ae-a6c5956cf139 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:18.967390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:18.968316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:18.968965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:18.969377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] PATH_INFO: `/v3/users/786721d4cca643d2839b3500dd3849d9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:18.970909 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=786721d4cca643d2839b3500dd3849d9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:18.973155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:19.037133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786721d4cca643d2839b3500dd3849d9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.055367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786721d4cca643d2839b3500dd3849d9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.056465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786721d4cca643d2839b3500dd3849d9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.066447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786721d4cca643d2839b3500dd3849d9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.074393 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786721d4cca643d2839b3500dd3849d9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.080620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1301/2580] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:18 2022] DELETE /identity/v3/users/786721d4cca643d2839b3500dd3849d9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:19.296450 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:19.317523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f9c7a-6465-415e-9c15-d1c2ea68fb6a service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:19.389012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:19.389911 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:19.390195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:19.390477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] PATH_INFO: `/v3/projects/e7753d5879514268b8ad39e1b1b6dd3b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:19.391699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7753d5879514268b8ad39e1b1b6dd3b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:19.393780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:19.445829 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7753d5879514268b8ad39e1b1b6dd3b', 'request_id': 'req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.454542 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7753d5879514268b8ad39e1b1b6dd3b', 'request_id': 'req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.461078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7753d5879514268b8ad39e1b1b6dd3b', 'request_id': 'req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.472913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] The token cache is being invalidate because project e7753d5879514268b8ad39e1b1b6dd3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:19.473206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.473861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.475488 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1280/2581] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:19 2022] DELETE /identity/v3/projects/e7753d5879514268b8ad39e1b1b6dd3b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:19.480010 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:19.481557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:19.495548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24235d1c-ae74-4adf-83d6-a15e4113e653 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:19.529826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:19.560046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:19.560940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:19.561248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:19.561592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] PATH_INFO: `/v3/users/b885586eb931442c968a7b3c1e6826f0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:19.562826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b885586eb931442c968a7b3c1e6826f0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:19.564845 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:19.618875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b885586eb931442c968a7b3c1e6826f0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.631868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b885586eb931442c968a7b3c1e6826f0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.632615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b885586eb931442c968a7b3c1e6826f0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.642956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b885586eb931442c968a7b3c1e6826f0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.651858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b8eb0dd7-ea66-42ed-8c78-03b896810302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b885586eb931442c968a7b3c1e6826f0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.658584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1302/2582] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:19 2022] DELETE /identity/v3/users/b885586eb931442c968a7b3c1e6826f0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:19.846848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:19.848412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:19.861934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c16e7-d21f-47df-b2b4-f39e6888b3fd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:19.922762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:19.923697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:19.923965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:19.924243 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] PATH_INFO: `/v3/projects/18cf7994731a433daa60dd220c293a3c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:19.925427 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cf7994731a433daa60dd220c293a3c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:19.927768 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:19.953824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:19.982478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cf7994731a433daa60dd220c293a3c', 'request_id': 'req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:19.991727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cf7994731a433daa60dd220c293a3c', 'request_id': 'req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:20.000838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cf7994731a433daa60dd220c293a3c', 'request_id': 'req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:20.013285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] The token cache is being invalidate because project 18cf7994731a433daa60dd220c293a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:20.013574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:20.014280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e46139bb-cc35-43a8-bb14-d88b3b8a1e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:20.015967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1281/2583] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:19 2022] DELETE /identity/v3/projects/18cf7994731a433daa60dd220c293a3c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:20.021076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.021550 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.021650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.040286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:20.041784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:20.108723 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1303/2584] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:20.116921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.117333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.117773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.141165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:20.142379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:20.151203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1282/2585] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:20.155581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.166340 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716f77d-dc44-4e4b-9a32-81f72e0d12c4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.170978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.171662 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.171940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.172212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.172932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:20.173235 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.174851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.187047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1304/2586] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:48:20.191982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.204001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3698bd-aef9-45b7-899c-aa736fa928ce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.213042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.213949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.214307 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.214569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.215306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.218284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.261922 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1283/2587] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:20.265965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.276731 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442d70-f63a-4b19-aaf5-a80e6ba54796 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.316706 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:20.338097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.338794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.339099 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.339385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.340089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.342045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.363966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1305/2588] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:20.367916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.378581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ada3-ddac-4e73-8bf4-674b725325e2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.383383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.384063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.384357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.384627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.385363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.386987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.397297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1284/2589] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:20.401430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.411899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d702af-3c72-4b3c-a85c-eb763c98202f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.416814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.417562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.417921 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.418274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.419125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.420811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f53e2-b72a-4734-878a-e4bbf535d05c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.435839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1306/2590] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:20.437077 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.448684 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f513c7c-35c2-4a2f-bb9e-58a5839d9ffb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.453603 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.454532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.454812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.455100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] PATH_INFO: `/v3/projects/571a39eee3dc4748ad844cc2ade207db/users/7df473794c3b4ad99319f6e923388efe/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.471259 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=571a39eee3dc4748ad844cc2ade207db, user_id=7df473794c3b4ad99319f6e923388efe, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.474014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.488635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1285/2591] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:20 2022] PUT /identity/v3/projects/571a39eee3dc4748ad844cc2ade207db/users/7df473794c3b4ad99319f6e923388efe/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:20.500068 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.500425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.500766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.528946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] MFA Rules not processed for user `7df473794c3b4ad99319f6e923388efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:20.530159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:20.605144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1307/2592] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:20.610569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.675577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f955a-a248-4c80-93bf-7e59134eec06 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.681000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.681671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.681956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.682232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.682925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.684868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.712424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1286/2593] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:20.716607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.727810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d5fa-007a-4034-9a56-9c6c189a3e87 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.787577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.788266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.788549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.788824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.789584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.791589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.819025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1308/2594] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:20 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:48:20.825728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.839590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e273-723c-4585-8d63-6200cd57b915 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.854695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.856130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.856404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.856712 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.857448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.859060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.871594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1287/2595] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:20.875743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.886958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38205184-3fdb-4b75-a4f2-8da066b0d0b9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.892055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:20.893143 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:20.893447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:20.893727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] PATH_INFO: `/v3/projects/8a177f3f86c84bf1990d34c3f5843a53/users/2c166a6646884c0bb7c71e94bb1a733e/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:20.910316 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a177f3f86c84bf1990d34c3f5843a53, user_id=2c166a6646884c0bb7c71e94bb1a733e, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:20.913265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:20.927904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1309/2596] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:20 2022] PUT /identity/v3/projects/8a177f3f86c84bf1990d34c3f5843a53/users/2c166a6646884c0bb7c71e94bb1a733e/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:20.931913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:20.942992 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5a82b-8b23-46f2-a2d8-2811e54e34a5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:20.955587 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:21.010199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:21.010882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:21.011195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:21.011476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:21.012206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:21.013826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbcd2a-5151-472f-8fca-13c4ea89ba79 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:21.023722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1288/2597] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:21.028884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:21.041811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f52b2d-23ee-4561-900b-25e321f99338 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:21.047281 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:21.048210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:21.048494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:21.048767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] PATH_INFO: `/v3/projects/8a177f3f86c84bf1990d34c3f5843a53/users/2c166a6646884c0bb7c71e94bb1a733e/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:21.050904 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a177f3f86c84bf1990d34c3f5843a53, user_id=2c166a6646884c0bb7c71e94bb1a733e, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:21.053625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:21.060319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:21.065338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1310/2598] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:21 2022] PUT /identity/v3/projects/8a177f3f86c84bf1990d34c3f5843a53/users/2c166a6646884c0bb7c71e94bb1a733e/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:21.076231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:21.076524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:21.076811 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:21.104734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] MFA Rules not processed for user `2c166a6646884c0bb7c71e94bb1a733e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:21.106143 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:21.184936 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1289/2599] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:48:21.196007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:21.198113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:21.263271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed39f0-8578-4f05-9586-f729f0026e1b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:21.270335 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:21.271007 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:21.271319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:21.271588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:21.337943 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:21.339444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af617a-966e-4792-bb64-ef92700331fd service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:21.349245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1311/2600] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:25.667499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:25.677211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88b6fb-d93f-4c27-bcea-924f79c27c49 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:25.681257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:25.681884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:25.682167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:25.682439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:25.692300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:25.694382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:25.704327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1290/2601] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:26.816495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:26.816495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:26.816495 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:26.834560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] MFA Rules not processed for user `faa76f7a5fa04d9ca1d2489cc12e9e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:26.835826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.895705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1312/2602] 10.222.0.65 () {58 vars in 967 bytes} [Fri Jun 24 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:26.908030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:26.909524 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:26.989784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dcf4c-b0d4-4da1-aa28-39cb04f5e562 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:26.995254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '616b28448ac548c0b9c22f1245b090b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:26.995919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:26.996205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:26.996485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:27.005060 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:27.006666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2301378-4cc5-4280-8f24-715c06f4eb50 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:27.015109 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1291/2603] 10.222.0.65 () {60 vars in 1384 bytes} [Fri Jun 24 09:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 09:48:30.637255 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:30.725176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d91ad-bd94-4856-8e07-afd13dcccd01 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:30.731494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:30.732435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:30.732716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:30.732997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] PATH_INFO: `/v3/users/7df473794c3b4ad99319f6e923388efe` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:30.734224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df473794c3b4ad99319f6e923388efe)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:30.736212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:30.737832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:30.789753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df473794c3b4ad99319f6e923388efe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:30.805011 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df473794c3b4ad99319f6e923388efe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:30.805740 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df473794c3b4ad99319f6e923388efe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:30.814299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df473794c3b4ad99319f6e923388efe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:30.823126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df473794c3b4ad99319f6e923388efe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:30.830202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1313/2604] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:30 2022] DELETE /identity/v3/users/7df473794c3b4ad99319f6e923388efe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:30.834860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:30.835396 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:30.835494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:30.853812 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:30.854966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:30.860344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:30.917668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1292/2605] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:30.928243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:30.929724 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:31.000748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ffce0-ed8e-44dd-9849-963489383188 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:31.008648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:31.009395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:31.009773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:31.010220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:31.029701 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:31.031219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:31.042672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1314/2606] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:31.328469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:31.330645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:31.343650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61abc09-52ec-4f13-9436-733e949808d8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:31.349464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:31.350406 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:31.350693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:31.350970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] PATH_INFO: `/v3/projects/571a39eee3dc4748ad844cc2ade207db` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:31.352223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] RBAC: Authorizing `identity:delete_project(project_id=571a39eee3dc4748ad844cc2ade207db)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:31.354192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:31.410277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '571a39eee3dc4748ad844cc2ade207db', 'request_id': 'req-8373b608-ef53-4af3-80d9-027fdcc8422c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.422153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '571a39eee3dc4748ad844cc2ade207db', 'request_id': 'req-8373b608-ef53-4af3-80d9-027fdcc8422c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.436388 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '571a39eee3dc4748ad844cc2ade207db', 'request_id': 'req-8373b608-ef53-4af3-80d9-027fdcc8422c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.448828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] The token cache is being invalidate because project 571a39eee3dc4748ad844cc2ade207db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:31.449223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.450147 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.452116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1293/2607] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:31 2022] DELETE /identity/v3/projects/571a39eee3dc4748ad844cc2ade207db => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:31.456037 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:31.466424 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959bc66-6651-48a1-bc02-86132cab6854 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:31.526596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:31.527533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:31.527817 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:31.528305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] PATH_INFO: `/v3/users/2c166a6646884c0bb7c71e94bb1a733e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:31.529544 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c166a6646884c0bb7c71e94bb1a733e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:31.531524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:31.555915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:31.585251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c166a6646884c0bb7c71e94bb1a733e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.600527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c166a6646884c0bb7c71e94bb1a733e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.601216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c166a6646884c0bb7c71e94bb1a733e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.609260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c166a6646884c0bb7c71e94bb1a733e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.616193 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-923c48a5-8647-4c08-8361-6d206bd5df9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c166a6646884c0bb7c71e94bb1a733e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:31.622249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1315/2608] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:31 2022] DELETE /identity/v3/users/2c166a6646884c0bb7c71e94bb1a733e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:31.909559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:31.911162 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:31.924907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373b608-ef53-4af3-80d9-027fdcc8422c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:31.986654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:31.987491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:31.987775 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:31.988052 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] PATH_INFO: `/v3/projects/8a177f3f86c84bf1990d34c3f5843a53` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:31.989287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a177f3f86c84bf1990d34c3f5843a53)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:31.991320 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.034362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:32.045233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a177f3f86c84bf1990d34c3f5843a53', 'request_id': 'req-3c12053b-d016-495a-8a9a-3a61fd1492bb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.053316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a177f3f86c84bf1990d34c3f5843a53', 'request_id': 'req-3c12053b-d016-495a-8a9a-3a61fd1492bb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.060142 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a177f3f86c84bf1990d34c3f5843a53', 'request_id': 'req-3c12053b-d016-495a-8a9a-3a61fd1492bb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.074322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] The token cache is being invalidate because project 8a177f3f86c84bf1990d34c3f5843a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:32.074566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.075790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c12053b-d016-495a-8a9a-3a61fd1492bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.077588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1294/2609] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:31 2022] DELETE /identity/v3/projects/8a177f3f86c84bf1990d34c3f5843a53 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:32.083857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.084400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.084400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.104477 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:32.105799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:32.178880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1316/2610] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:32.188520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.188813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.189096 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.208554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:32.209782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:32.219619 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1295/2611] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:32.224063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.234628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d6827-f412-43b0-92b7-096a54f790bc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.240252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.240856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.241142 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.241420 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.242151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:32.242464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.244132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.256626 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1317/2612] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:48:32.263128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.272704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a90dd-07fa-4dde-976f-4b40d64d6b00 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.279054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.279778 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.280065 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.280349 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.281055 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.282975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.316507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1296/2613] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:48:32.321246 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.332967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5270f-c5f0-4af8-a2f1-12555a61f8d3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.393325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.393966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.394253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.394532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.395267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.397220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.424013 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1318/2614] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:32.428334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.440434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d641e-6021-4772-879e-4f7479916abd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.446327 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.447192 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.447478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.447755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.448516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.450332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.461175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1297/2615] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:32.465681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.481897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e91c7-e6d8-442f-9587-e73d741ca86d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.487779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.488741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.489095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.489456 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.490634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.492945 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853bc75-751a-42bf-a1df-553847a166d1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.504463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1319/2616] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:32.508697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.519683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8141-d9bb-40b1-a056-2073a1c2784d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.525324 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.526219 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.526504 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.526779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] PATH_INFO: `/v3/projects/e81214f942784b63a0f3483c9877d3a9/users/e9e46c1e9f46494fa6748be7dec1afe6/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.541723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81214f942784b63a0f3483c9877d3a9, user_id=e9e46c1e9f46494fa6748be7dec1afe6, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.544444 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.559625 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1298/2617] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:32 2022] PUT /identity/v3/projects/e81214f942784b63a0f3483c9877d3a9/users/e9e46c1e9f46494fa6748be7dec1afe6/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:32.570987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.571311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.571627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.590036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:32.607006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] MFA Rules not processed for user `e9e46c1e9f46494fa6748be7dec1afe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:32.608263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:32.676986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1320/2618] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:32.688528 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.690306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:32.758011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fcd42-2f07-4b43-903c-b216f45eef9a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.767637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.768314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.768602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.768883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.778251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.779755 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9628a-fea3-4ed4-8627-04d0b5bb2df7 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.788773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1299/2619] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:32.809578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:32.885275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc029e4-f087-4077-ac88-47214866dd08 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:32.890994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:32.891940 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.892218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.892490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] PATH_INFO: `/v3/users/e9e46c1e9f46494fa6748be7dec1afe6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:32.893687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e46c1e9f46494fa6748be7dec1afe6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:32.895886 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:32.946485 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e46c1e9f46494fa6748be7dec1afe6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.958637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e46c1e9f46494fa6748be7dec1afe6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.959356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e46c1e9f46494fa6748be7dec1afe6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.969169 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e46c1e9f46494fa6748be7dec1afe6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.976155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e46c1e9f46494fa6748be7dec1afe6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:32.982925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1321/2620] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:32 2022] DELETE /identity/v3/users/e9e46c1e9f46494fa6748be7dec1afe6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:32.987921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:32.988404 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:32.988517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.007066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:33.008242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:33.070125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1300/2621] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:33.080408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.082071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:33.102408 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:33.143710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6de20-f660-43f4-b863-3c1544c07ec6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.147388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.148059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.148336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.148613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.158790 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.160396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ddc46-edfc-4040-8ea3-6eed2db40426 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.170247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1322/2622] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:33.440569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.442137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:33.450724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:33.455656 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce927d0f-d9d4-4652-9c48-644d5dafe485 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.461675 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.462627 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.462913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.463234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] PATH_INFO: `/v3/projects/e81214f942784b63a0f3483c9877d3a9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.465346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81214f942784b63a0f3483c9877d3a9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.469016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.535495 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81214f942784b63a0f3483c9877d3a9', 'request_id': 'req-a28d9e04-feee-4176-9e77-bcad5afddcc9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:33.544247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81214f942784b63a0f3483c9877d3a9', 'request_id': 'req-a28d9e04-feee-4176-9e77-bcad5afddcc9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:33.550688 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81214f942784b63a0f3483c9877d3a9', 'request_id': 'req-a28d9e04-feee-4176-9e77-bcad5afddcc9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:33.562765 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] The token cache is being invalidate because project e81214f942784b63a0f3483c9877d3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:33.563115 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:33.563938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a28d9e04-feee-4176-9e77-bcad5afddcc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:33.565974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1301/2623] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:33 2022] DELETE /identity/v3/projects/e81214f942784b63a0f3483c9877d3a9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:33.571363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.571675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.571954 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.591460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:33.592805 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:33.660598 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1323/2624] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:33.668494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.669418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.669716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.690704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:33.691940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:33.701369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1302/2625] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:33.705859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.716671 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8076f62-b65d-4e4e-a2b7-fc3950e3f78d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.722132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.722884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.723267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.723634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.724514 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:33.724821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.726460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.740251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1324/2626] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:48:33.748369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.761551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a379a-8821-4a39-87b3-c84feb8a1057 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.769212 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.769951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.770553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.770824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.771561 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.773496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.813721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1303/2627] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:33 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:33.818388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.831865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ace81d-af1b-4ad7-9266-e76417ffd720 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.901239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.901935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.902219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.902498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.903289 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.905253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.931124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1325/2628] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:33 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:33.932857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.944405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f277-e773-40a6-b4e0-835f1b46bb27 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.951850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.952559 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.952846 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.953130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.953881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.955584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:33.970921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1304/2629] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:33.972211 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:33.983187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e4d5-1497-45f4-8bfe-c9662df9119a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:33.988113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:33.988899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:33.989249 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:33.989594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:33.990402 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:33.992134 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f67000-c42d-48ca-a799-0d480762babd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:34.006551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1326/2630] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:48:34.008469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:34.022148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502c7c-3a70-4fb8-b138-59bb75b69577 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:34.027730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:34.031126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:34.031126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:34.031126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] PATH_INFO: `/v3/projects/86f042cb88b7426ea9733f70033b55e3/users/565cc9705a94454b9e68595b85c6a286/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:34.048328 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f042cb88b7426ea9733f70033b55e3, user_id=565cc9705a94454b9e68595b85c6a286, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:34.051316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:34.071000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1305/2631] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:34 2022] PUT /identity/v3/projects/86f042cb88b7426ea9733f70033b55e3/users/565cc9705a94454b9e68595b85c6a286/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:34.083789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a51f-df07-4737-b5d4-53110b52d91c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:34.084290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a51f-df07-4737-b5d4-53110b52d91c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:34.084507 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906a51f-df07-4737-b5d4-53110b52d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:34.126358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1906a51f-df07-4737-b5d4-53110b52d91c None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:34.128050 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1906a51f-df07-4737-b5d4-53110b52d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:34.152230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:34.231137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1327/2632] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:34.241436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:34.243264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:34.303994 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79591-a985-4f84-a1d2-b08375ece6d6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:34.307578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:34.308236 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:34.308519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:34.308810 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:34.319932 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:34.321426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:34.331807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1306/2633] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:38.582097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:38.582674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:38.582674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:38.611008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] MFA Rules not processed for user `565cc9705a94454b9e68595b85c6a286`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:38.612325 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:38.690476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1328/2634] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:38.695138 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:38.705206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a145-1d30-4f94-bb3b-9a0ee943920e service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:38.710631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:38.711334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:38.711622 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:38.711898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:38.712602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:38.714578 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:38.749235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1307/2635] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:38 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:38.753447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:38.765976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131bf9a-5a33-4650-a5bd-40efe2863c60 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:38.824392 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:38.825354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:38.825640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:38.825920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:38.826614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:38.828596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:38.868618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1329/2636] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:38 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:38.870165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:38.878123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:38.882408 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36261566-6d07-48d4-923a-e9ad876ff3be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:38.894616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:38.895588 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:38.896222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:38.896572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:38.897548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:38.899489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:38.911003 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1308/2637] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:38.915359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:38.926002 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d381-8855-4047-b167-e6771e8fc003 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:38.930694 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:38.931638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:38.931931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:38.932206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] PATH_INFO: `/v3/projects/896af774f0b14c7d8727c344e41537f8/users/7568764c93b54e2496c947ed049f56af/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:38.948856 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896af774f0b14c7d8727c344e41537f8, user_id=7568764c93b54e2496c947ed049f56af, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:38.951854 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:38.969823 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1330/2638] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:38 2022] PUT /identity/v3/projects/896af774f0b14c7d8727c344e41537f8/users/7568764c93b54e2496c947ed049f56af/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:48:38.974071 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:38.985206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239191a3-ba94-4932-bd4a-0763290354cc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:39.044239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:39.044935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:39.045225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:39.045505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:39.046241 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:39.047955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:39.058602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1309/2639] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:39.062964 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:39.073637 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbd0c5-1ddc-4bb6-bf21-b1e9922f54b8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:39.078769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:39.079721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:39.080004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:39.080278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] PATH_INFO: `/v3/projects/896af774f0b14c7d8727c344e41537f8/users/7568764c93b54e2496c947ed049f56af/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:39.082298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] RBAC: Authorizing `identity:create_grant(project_id=896af774f0b14c7d8727c344e41537f8, user_id=7568764c93b54e2496c947ed049f56af, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:39.084994 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:39.097918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1331/2640] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:39 2022] PUT /identity/v3/projects/896af774f0b14c7d8727c344e41537f8/users/7568764c93b54e2496c947ed049f56af/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:39.637717 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:39.709245 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862814d8-fb94-45e0-b810-6035522622ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:39.801070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:39.801773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:39.802053 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:39.802329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:39.809956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:39.911497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:39.914792 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:39.916530 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ceab6d-20d1-4d2e-bc8e-50b59f153a65 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:39.936691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1310/2641] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:46.387124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:46.389120 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:46.402754 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4207f2-6cc1-4ee6-b98d-d28f8729970f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:46.408607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:46.409685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:46.410051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:46.410411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] PATH_INFO: `/v3/users/fd136fcda9a04a16b258301a86bde8ef` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:46.411902 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd136fcda9a04a16b258301a86bde8ef)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:46.414054 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:46.418147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:46.479707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd136fcda9a04a16b258301a86bde8ef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.494937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd136fcda9a04a16b258301a86bde8ef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.496471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd136fcda9a04a16b258301a86bde8ef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.510766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd136fcda9a04a16b258301a86bde8ef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.520122 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd136fcda9a04a16b258301a86bde8ef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.527708 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1332/2642] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:46 2022] DELETE /identity/v3/users/fd136fcda9a04a16b258301a86bde8ef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:46.677872 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:46.678474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:46.678714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:46.711928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] MFA Rules not processed for user `7568764c93b54e2496c947ed049f56af`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:46.713467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:46.729484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:46.731236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:46.806148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1311/2643] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:46.817543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf63d1a-a007-4ce0-9ab5-fbdef0a3f638 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:46.822211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:46.824485 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:46.825946 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:46.826850 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:46.827176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:46.827449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] PATH_INFO: `/v3/projects/30033ce42b5a43b78dde38163831cea2` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:46.828743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30033ce42b5a43b78dde38163831cea2)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:46.830769 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:46.887587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30033ce42b5a43b78dde38163831cea2', 'request_id': 'req-6d972597-d8ea-45d1-98ca-bf95fdc69f26'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.894666 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30033ce42b5a43b78dde38163831cea2', 'request_id': 'req-6d972597-d8ea-45d1-98ca-bf95fdc69f26'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.904443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30033ce42b5a43b78dde38163831cea2', 'request_id': 'req-6d972597-d8ea-45d1-98ca-bf95fdc69f26'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.915771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e830d-066e-4f8a-a9c1-d03d98b460ae None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:46.917077 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] The token cache is being invalidate because project 30033ce42b5a43b78dde38163831cea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:46.917375 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.918214 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6d972597-d8ea-45d1-98ca-bf95fdc69f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:46.920206 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1333/2644] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:46 2022] DELETE /identity/v3/projects/30033ce42b5a43b78dde38163831cea2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:46.925164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7788-1425-441a-aa77-75283cf42afe None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:46.925463 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7788-1425-441a-aa77-75283cf42afe None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:46.925757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba7788-1425-441a-aa77-75283cf42afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:46.949923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-eeba7788-1425-441a-aa77-75283cf42afe None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:46.951433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eeba7788-1425-441a-aa77-75283cf42afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:46.957936 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:46.988773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:46.989522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:46.989900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:46.990277 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:46.992268 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:47.007341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:47.020210 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1334/2645] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:47.029894 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.030869 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.031225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.052356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:47.053609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:47.063647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1335/2646] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:47.067395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.071730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.073698 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.078000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb70c92-00c7-4510-bbca-53b73165514d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.085884 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.086761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1312/2647] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:47.086897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.087762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.087762 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.088369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:47.088687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.090432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.103578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1336/2648] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:48:47.108233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.118727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b0395-e7d9-463e-a5b5-9c5b7829487d service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.124403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.125155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.125536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.125816 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.126418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.129339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.162061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1313/2649] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:47 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:48:47.166726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.183123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd27cd6-163e-4a44-b332-396b6ef1ae17 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.248970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.249668 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.249956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.250239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.250956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.253005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.278958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1337/2650] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:47 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:47.283805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.295238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db73bc-cf2b-4a31-9dc6-bc4d2a8c6e26 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.305298 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.305985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.306319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.306599 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.307385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.309491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.328632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1314/2651] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:48:47.332436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.344396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736394c-f8f4-4b71-bbcb-5ba3b0aefc04 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.349760 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.350443 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.351297 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.351581 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.352363 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.354049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b6e7f-3248-413c-9d0a-b37fadf20170 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.368625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1338/2652] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:47.370014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.381814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eefa17-0745-4938-a99b-f2b9fc2144ab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.387203 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.388786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.389139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.389420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] PATH_INFO: `/v3/projects/270cf0d110eb4e8fa107a4cf76b09691/users/5a18b06cd76a4edebeb4085623969068/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.407750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] RBAC: Authorizing `identity:create_grant(project_id=270cf0d110eb4e8fa107a4cf76b09691, user_id=5a18b06cd76a4edebeb4085623969068, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.410512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.426962 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1315/2653] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:47 2022] PUT /identity/v3/projects/270cf0d110eb4e8fa107a4cf76b09691/users/5a18b06cd76a4edebeb4085623969068/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:47.437559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.438024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.438024 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.475257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] MFA Rules not processed for user `5a18b06cd76a4edebeb4085623969068`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:47.476614 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:47.572645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:47.581109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1339/2654] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:47.594118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:47.596836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:47.735653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4de4-d870-43d3-b66a-c9e3b660633b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:47.744790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:47.746228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:47.746696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:47.747116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:47.765491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:47.768238 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c91850-433d-49a5-a819-63198ded9521 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:47.790888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1316/2655] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:55.404488 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:55.470008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813df06e-3b98-4df3-8db2-bddad592e588 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:55.474582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:55.475522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:55.475808 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:55.476088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] PATH_INFO: `/v3/users/5a18b06cd76a4edebeb4085623969068` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:55.477294 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a18b06cd76a4edebeb4085623969068)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:55.479272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:55.548610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a18b06cd76a4edebeb4085623969068'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:55.567716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a18b06cd76a4edebeb4085623969068'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:55.568398 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a18b06cd76a4edebeb4085623969068'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:55.577696 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a18b06cd76a4edebeb4085623969068'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:55.585604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a18b06cd76a4edebeb4085623969068'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:55.591742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1340/2656] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:55 2022] DELETE /identity/v3/users/5a18b06cd76a4edebeb4085623969068 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:55.597188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:55.597713 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:55.597809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:55.616085 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:55.617306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:55.660332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:55.680431 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1317/2657] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:55.690502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:55.692067 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:55.757088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038abe0-e520-4ec9-85db-7e657d026cbb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:55.761465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:55.762582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:55.762982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:55.763295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:55.775096 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:55.776577 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1031c6e-64f5-4768-8667-24ff463abcf9 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:55.786806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1341/2658] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:56.071894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.073425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:56.087373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79041af7-cdd1-404d-bd1d-cf49179605cc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.092831 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.094508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.095150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.095433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] PATH_INFO: `/v3/projects/270cf0d110eb4e8fa107a4cf76b09691` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.096796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=270cf0d110eb4e8fa107a4cf76b09691)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.098596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.148089 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '270cf0d110eb4e8fa107a4cf76b09691', 'request_id': 'req-52b7692e-21b4-44b8-8a6e-6efdf7468d51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:56.156960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '270cf0d110eb4e8fa107a4cf76b09691', 'request_id': 'req-52b7692e-21b4-44b8-8a6e-6efdf7468d51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:56.162939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '270cf0d110eb4e8fa107a4cf76b09691', 'request_id': 'req-52b7692e-21b4-44b8-8a6e-6efdf7468d51'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:56.177948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] The token cache is being invalidate because project 270cf0d110eb4e8fa107a4cf76b09691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:56.178248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:56.179025 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-52b7692e-21b4-44b8-8a6e-6efdf7468d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:56.181759 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1318/2659] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:56 2022] DELETE /identity/v3/projects/270cf0d110eb4e8fa107a4cf76b09691 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:56.186395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.186647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.186935 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.204999 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:56.206219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:56.228593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:56.276540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1342/2660] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:56.285608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.285900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.286183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.305222 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:56.306428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:56.315400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1319/2661] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:56.320030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.330153 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42b048-2cf6-43b3-be0a-63b895707b96 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.335971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.336648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.336932 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.337209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.337933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:56.338244 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.339901 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.355354 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1343/2662] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:48:56.361346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.375322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f544d1-c636-4503-8bba-0a073b742d9b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.380459 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.381116 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.381395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.381672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.382365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.384312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.412736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1320/2663] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:56.417434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.429049 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a9984-cd4d-49ab-b3c7-c46ef20bbb89 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.488396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.489076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.489358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.489630 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.490317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.492277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.514862 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1344/2664] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:56.519817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.533144 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772fefe-5cf9-4211-869a-c6d217eb7a17 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.539835 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.540589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.540896 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.541182 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.541955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.543941 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.556849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1321/2665] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:56.561226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.573394 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977601b-8486-4290-8956-bf34a1560610 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.580865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.581566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.581851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.582124 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.583259 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.585700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fbd05-29df-4b9d-8f60-0b87a7f06d15 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.603026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1345/2666] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:56.609284 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.625626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4308fc1-7819-46cd-8b76-a9e930698f72 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.634368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.635884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.636310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.636797 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] PATH_INFO: `/v3/projects/f2f2aa0a18754954b9b74251e4bdb026/users/730b13e470124a3586dfe4320fd960f4/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.653291 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f2aa0a18754954b9b74251e4bdb026, user_id=730b13e470124a3586dfe4320fd960f4, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.656555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.672105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1322/2667] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:56 2022] PUT /identity/v3/projects/f2f2aa0a18754954b9b74251e4bdb026/users/730b13e470124a3586dfe4320fd960f4/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:56.682866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.683202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.683499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.718792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] MFA Rules not processed for user `730b13e470124a3586dfe4320fd960f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:56.720238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:56.788831 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1346/2668] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:56.793630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.861022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0b4a-17bd-425b-a108-f8d7b22321dc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.867220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.868043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.868393 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.868730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.869499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.871501 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:56.882049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:56.904155 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1323/2669] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:56.908785 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:56.919729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd06eb5-51b8-4e2a-aab0-38587b452148 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:56.932421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:56.986984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:56.987699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:56.987976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:56.988258 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:56.988923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:56.990991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.018230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1347/2670] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:56 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:57.019682 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.030694 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05498-1710-4b7b-b68b-17f04d41c076 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.036901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.037575 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.037855 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.038128 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.038857 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.040507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.051597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1324/2671] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:57.055974 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.067277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78057b-2f33-48e1-8527-aa4cb02ae36b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.073360 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.074494 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.074910 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.075301 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] PATH_INFO: `/v3/projects/056c7489e53e465a94ad9cec034f837d/users/5580392f755e496da7d4c96f3f759b4f/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.092357 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056c7489e53e465a94ad9cec034f837d, user_id=5580392f755e496da7d4c96f3f759b4f, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.095412 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.110378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1348/2672] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:57 2022] PUT /identity/v3/projects/056c7489e53e465a94ad9cec034f837d/users/5580392f755e496da7d4c96f3f759b4f/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:57.114503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.125119 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38754d6b-89bf-4821-bdf8-dd0f2e1af7b9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.185860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.186461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.186908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.187280 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.188018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.189647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77d24b-1fd0-4fef-8628-ca3c2fcce054 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.199643 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1325/2673] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:57.204203 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.220251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c640b0-8d69-484d-973a-e99b0fc124c4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.227359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.228715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.229150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.229546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] PATH_INFO: `/v3/projects/056c7489e53e465a94ad9cec034f837d/users/5580392f755e496da7d4c96f3f759b4f/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.232635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056c7489e53e465a94ad9cec034f837d, user_id=5580392f755e496da7d4c96f3f759b4f, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.237540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.253093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1349/2674] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:57 2022] PUT /identity/v3/projects/056c7489e53e465a94ad9cec034f837d/users/5580392f755e496da7d4c96f3f759b4f/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:57.266366 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.266551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.266716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.300227 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] MFA Rules not processed for user `5580392f755e496da7d4c96f3f759b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:57.301856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:57.376437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1326/2675] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:57.391280 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.393432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:57.455338 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91a7f3-b0b8-46eb-b370-e0414bc1de58 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.459242 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.459908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.460186 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.460461 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.471158 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.472645 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf5564-02b4-49fb-a53f-53de8ae58e27 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.483160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1350/2676] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:57.552813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.623347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8906e8c-e98b-48c1-828c-840d29bdefcf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.629027 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.629946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.630229 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.630498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] PATH_INFO: `/v3/users/730b13e470124a3586dfe4320fd960f4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.632036 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730b13e470124a3586dfe4320fd960f4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.634647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.636873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:57.688152 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730b13e470124a3586dfe4320fd960f4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:57.695572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730b13e470124a3586dfe4320fd960f4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:57.704702 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730b13e470124a3586dfe4320fd960f4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:57.705500 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730b13e470124a3586dfe4320fd960f4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:57.706118 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730b13e470124a3586dfe4320fd960f4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:57.724465 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1327/2677] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:57 2022] DELETE /identity/v3/users/730b13e470124a3586dfe4320fd960f4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:57.729305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.729602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.729908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.752792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:57.754319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:57.820202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1351/2678] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:57.830933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:57.832567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:57.898930 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d0be4-f318-430a-87b2-898ae6a41858 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:57.902971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:57.903668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:57.903956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:57.904230 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:57.914206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:57.915687 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:57.925377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1328/2679] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:48:58.222765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:58.224515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:58.241735 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6acd-a952-4f00-b6c8-e3be60ae4bdb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:58.248537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:58.249634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:58.249822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:58.250131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] PATH_INFO: `/v3/projects/f2f2aa0a18754954b9b74251e4bdb026` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:58.251568 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f2aa0a18754954b9b74251e4bdb026)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:58.253538 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:58.260530 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:58.307802 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f2aa0a18754954b9b74251e4bdb026', 'request_id': 'req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.314804 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f2aa0a18754954b9b74251e4bdb026', 'request_id': 'req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.325063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f2aa0a18754954b9b74251e4bdb026', 'request_id': 'req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.337606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] The token cache is being invalidate because project f2f2aa0a18754954b9b74251e4bdb026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:58.337923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.338592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.344367 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1352/2680] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:58 2022] DELETE /identity/v3/projects/f2f2aa0a18754954b9b74251e4bdb026 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:58.345721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:58.356455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555ca51-b382-4bea-a99d-2334f56838db service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:58.414339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:58.415269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:58.415548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:58.415777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] PATH_INFO: `/v3/users/5580392f755e496da7d4c96f3f759b4f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:58.417070 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5580392f755e496da7d4c96f3f759b4f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:58.419139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:58.444379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:58.471722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5580392f755e496da7d4c96f3f759b4f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.478507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5580392f755e496da7d4c96f3f759b4f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.487375 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5580392f755e496da7d4c96f3f759b4f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.488405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5580392f755e496da7d4c96f3f759b4f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.489175 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-196c73a2-0526-4631-ba27-c24b8e0e5686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5580392f755e496da7d4c96f3f759b4f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.512673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1329/2681] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:48:58 2022] DELETE /identity/v3/users/5580392f755e496da7d4c96f3f759b4f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:58.802482 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:58.804103 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:58.823421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb173-caa4-4c6c-a952-b3a5b24bb4ee admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:58.884717 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:58.885618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:58.885907 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:58.886185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] PATH_INFO: `/v3/projects/056c7489e53e465a94ad9cec034f837d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:58.887537 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056c7489e53e465a94ad9cec034f837d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:58.889652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:58.938816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056c7489e53e465a94ad9cec034f837d', 'request_id': 'req-39f5ffb7-27a0-445d-9c8d-e1389a01a597'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.945097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056c7489e53e465a94ad9cec034f837d', 'request_id': 'req-39f5ffb7-27a0-445d-9c8d-e1389a01a597'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.953674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056c7489e53e465a94ad9cec034f837d', 'request_id': 'req-39f5ffb7-27a0-445d-9c8d-e1389a01a597'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.964172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] The token cache is being invalidate because project 056c7489e53e465a94ad9cec034f837d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:48:58.964460 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.965105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-39f5ffb7-27a0-445d-9c8d-e1389a01a597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:48:58.966700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1353/2682] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:48:58 2022] DELETE /identity/v3/projects/056c7489e53e465a94ad9cec034f837d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:58.972422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:58.972982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:58.972982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:58.993309 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:58.994511 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:59.062579 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1330/2683] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:59.071986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.072257 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.072542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.092616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:59.093860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:59.102616 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1354/2684] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:59.107135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.117563 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89980199-33bf-4a29-88b6-cbfb06705dc1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.122858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.123564 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.123919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.124274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.125365 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:48:59.125673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.127358 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.140137 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1331/2685] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:48:59.144664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.154129 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4df9f-0d48-4052-91b7-76e7602510c0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.159370 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.160042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.160323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.160583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.161272 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.163208 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.178230 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:59.194202 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1355/2686] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:59.199288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.209630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff79f-e02f-4861-ae6f-bf66683269cf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.275377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.276098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.276382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.276657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.277344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.279345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.302100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1332/2687] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:59.306587 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.317938 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cccb7-a6b7-46d3-a3a2-5e52e51797b9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.323290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.324045 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.324226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.324638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.325428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.327345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.338913 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1356/2688] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:59.344012 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.354881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350374ce-cf72-4709-8275-2cb7f04eda5a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.360526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.361225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.361508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.361784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.362521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.364183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0744ad-e872-482d-98ac-56f0765b6c05 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.375347 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1333/2689] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:59.379987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.390656 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbff40c-53d7-4eae-842d-098df0085a13 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.395501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.396399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.396682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.396965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] PATH_INFO: `/v3/projects/c986e6ea788e48eca5466f5f9261f9ef/users/5e00f1c5e9a749c1ab6d2d7d17070033/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.412704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c986e6ea788e48eca5466f5f9261f9ef, user_id=5e00f1c5e9a749c1ab6d2d7d17070033, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.415925 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.429815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1357/2690] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:59 2022] PUT /identity/v3/projects/c986e6ea788e48eca5466f5f9261f9ef/users/5e00f1c5e9a749c1ab6d2d7d17070033/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:59.440517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.440881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.441228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.456276 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:48:59.472009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] MFA Rules not processed for user `5e00f1c5e9a749c1ab6d2d7d17070033`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:48:59.473217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:48:59.540198 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1334/2691] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:48:59.548838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.614775 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c8b0-83bc-4f84-8771-2690ae786905 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.620736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.621421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.621705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.621982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.622721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.624734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.652236 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1358/2692] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:59.656653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.667654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce57369-b7c6-42c6-837a-3ba43e79e0fe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.727395 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.728103 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.728389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.728668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.729357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.731323 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.752651 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1335/2693] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:48:59.756794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.767362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fe119-7bad-45e9-8d81-7819ba4654dc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.772194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.772876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.773167 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.773447 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.774178 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.775889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.787277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1359/2694] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:59.791718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.802585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc186b16-0ded-4cf7-b476-d9722cce8a8f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.807868 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.808771 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.809073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.809334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] PATH_INFO: `/v3/projects/07ce6bba236e4259a4a3181b8c705a3d/users/de4edec1d55545c1887c3ace4b9c1e5b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.826131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ce6bba236e4259a4a3181b8c705a3d, user_id=de4edec1d55545c1887c3ace4b9c1e5b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.829383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.844800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1336/2695] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:59 2022] PUT /identity/v3/projects/07ce6bba236e4259a4a3181b8c705a3d/users/de4edec1d55545c1887c3ace4b9c1e5b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:59.848920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.859834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec192c2-7e25-4ea4-9193-004be62917a7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.921615 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.922303 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.922612 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.922881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.923651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.925426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49769643-41ce-431f-9b1f-e2e01ccd6705 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.936358 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1360/2696] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:48:59.940912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:48:59.951956 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11915dc-f49e-480a-a741-29b036c62678 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:48:59.956863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:48:59.957753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.958032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.958300 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] PATH_INFO: `/v3/projects/07ce6bba236e4259a4a3181b8c705a3d/users/de4edec1d55545c1887c3ace4b9c1e5b/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:48:59.960305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ce6bba236e4259a4a3181b8c705a3d, user_id=de4edec1d55545c1887c3ace4b9c1e5b, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:48:59.962976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:48:59.974618 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1337/2697] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:48:59 2022] PUT /identity/v3/projects/07ce6bba236e4259a4a3181b8c705a3d/users/de4edec1d55545c1887c3ace4b9c1e5b/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:48:59.985764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:48:59.986062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:48:59.986344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:00.014291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] MFA Rules not processed for user `de4edec1d55545c1887c3ace4b9c1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:00.015741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:00.036009 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:00.088194 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1361/2698] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:00.099477 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:00.101445 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:00.158928 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:00.163058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef4742-d4ae-4f14-b8d8-8acd2e524e3d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:00.168570 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:00.169237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:00.169515 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:00.169803 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:00.179912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:00.181361 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3f962-8e06-4f7c-9f6a-22422f561774 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:00.191242 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1338/2699] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:00.362871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:00.377880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f97946-01f9-4a3b-a712-815a70dd5b50 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:00.383520 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:00.384449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:00.384777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:00.385070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:00.446755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:00.448434 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:00.458228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1362/2700] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:00 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:06.836432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863bef-637f-47fa-9b57-f6b005a2fa4c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:06.837491 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863bef-637f-47fa-9b57-f6b005a2fa4c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:06.838006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863bef-637f-47fa-9b57-f6b005a2fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:06.859611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-47863bef-637f-47fa-9b57-f6b005a2fa4c None None] MFA Rules not processed for user `565cc9705a94454b9e68595b85c6a286`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:06.860786 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-47863bef-637f-47fa-9b57-f6b005a2fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:06.918382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1339/2701] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:09.000889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:09.075421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649682a-35f4-4843-9849-9e1dfe55039b service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:09.080871 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:09.082625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:09.082917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:09.083226 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] PATH_INFO: `/v3/users/5e00f1c5e9a749c1ab6d2d7d17070033` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:09.084803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e00f1c5e9a749c1ab6d2d7d17070033)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:09.086737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:09.145991 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e00f1c5e9a749c1ab6d2d7d17070033'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.155893 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:09.165524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e00f1c5e9a749c1ab6d2d7d17070033'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.166500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e00f1c5e9a749c1ab6d2d7d17070033'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.176326 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e00f1c5e9a749c1ab6d2d7d17070033'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.185416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e00f1c5e9a749c1ab6d2d7d17070033'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.192673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1363/2702] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:08 2022] DELETE /identity/v3/users/5e00f1c5e9a749c1ab6d2d7d17070033 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:09.197150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:09.197747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:09.197747 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:09.217836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:09.219455 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:09.303594 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1340/2703] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:09.313782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:09.315484 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:09.376004 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13326-8cef-417e-b899-031c5995a8d6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:09.380635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:09.380635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:09.381025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:09.381304 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:09.393200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:09.394832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:09.419343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1364/2704] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:09.809849 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:09.813049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:09.827486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c54769-ec57-485b-882e-6d55b1e756f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:09.833316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:09.834214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:09.843363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:09.843664 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] PATH_INFO: `/v3/projects/c986e6ea788e48eca5466f5f9261f9ef` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:09.845171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c986e6ea788e48eca5466f5f9261f9ef)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:09.847507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:09.898605 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c986e6ea788e48eca5466f5f9261f9ef', 'request_id': 'req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.907616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c986e6ea788e48eca5466f5f9261f9ef', 'request_id': 'req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.913673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c986e6ea788e48eca5466f5f9261f9ef', 'request_id': 'req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.925813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] The token cache is being invalidate because project c986e6ea788e48eca5466f5f9261f9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:09.926261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.926923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:09.928526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1341/2705] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:09 2022] DELETE /identity/v3/projects/c986e6ea788e48eca5466f5f9261f9ef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:09.932657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:09.942896 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed939b-05d3-4494-8701-3e2d5ce87dcc service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.015111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.016025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.016311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.016591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] PATH_INFO: `/v3/users/de4edec1d55545c1887c3ace4b9c1e5b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.018006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4edec1d55545c1887c3ace4b9c1e5b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.020089 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.077501 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4edec1d55545c1887c3ace4b9c1e5b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.092970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4edec1d55545c1887c3ace4b9c1e5b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.093660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4edec1d55545c1887c3ace4b9c1e5b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.104499 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4edec1d55545c1887c3ace4b9c1e5b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.117364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:10.122640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0d5c1555-3fa9-4b3f-af9f-761a9132cad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4edec1d55545c1887c3ace4b9c1e5b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.132841 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1365/2706] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:09 2022] DELETE /identity/v3/users/de4edec1d55545c1887c3ace4b9c1e5b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:10.446471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:10.448913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:10.466997 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf98d7-495e-42b0-8d9b-dcbc185cc302 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.491858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:10.542308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.543336 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.543628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.543955 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] PATH_INFO: `/v3/projects/07ce6bba236e4259a4a3181b8c705a3d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.545335 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ce6bba236e4259a4a3181b8c705a3d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.547352 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.602832 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ce6bba236e4259a4a3181b8c705a3d', 'request_id': 'req-e8b7b872-8d69-48ed-ad98-09bed025d8c6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.611663 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ce6bba236e4259a4a3181b8c705a3d', 'request_id': 'req-e8b7b872-8d69-48ed-ad98-09bed025d8c6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.617475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ce6bba236e4259a4a3181b8c705a3d', 'request_id': 'req-e8b7b872-8d69-48ed-ad98-09bed025d8c6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.629156 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] The token cache is being invalidate because project 07ce6bba236e4259a4a3181b8c705a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:10.629484 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.630102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e8b7b872-8d69-48ed-ad98-09bed025d8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:10.632006 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1342/2707] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:10 2022] DELETE /identity/v3/projects/07ce6bba236e4259a4a3181b8c705a3d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:49:10.636809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.637132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.637389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.655489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:10.656715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:10.738897 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1366/2708] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:49:10.749478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.749478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.749478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.760053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:10.766629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:10.770964 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:10.772194 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:10.786282 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1343/2709] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:10.787865 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:10.799289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cb509-d88e-41be-8530-705ab72c98ab None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.805533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.806225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.806518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.807489 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.808248 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:49:10.808566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.810217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.824695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1344/2710] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:49:10.830957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:10.840704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b724db-9111-4e68-a561-c1e39e27504f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.845449 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b459-3d22-477b-ace8-bb01e3334dd8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.846502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.847578 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.847859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.848192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.849968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:10.850863 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.851632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.851923 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.852265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.852963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.854942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.865863 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.867509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.879825 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1367/2711] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:10.884111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1345/2712] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:10 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:10.889571 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:10.899438 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8164506-812f-424e-a43b-3cb8049e798c service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:10.957743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:10.958459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:10.958744 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:10.959019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:10.959758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:10.961751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:10.985271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1368/2713] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:10 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:10.989610 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.000938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e673f-26c5-4a7e-a71d-022ffc70fcf6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.005927 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.006612 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.006893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.007195 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.008102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.009877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.023151 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1346/2714] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:11.024875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.040151 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d126f-6f5b-42e0-9cbc-d27d6c7f9302 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.046277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.047098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.047464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.047822 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.048684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.050623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58dff9-b0bb-4ef6-8118-c2677dab96e5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.061759 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1369/2715] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:11.065736 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.076106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6a7-4c4a-4bd2-88db-89115edce583 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.082628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.083228 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.083513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.083791 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] PATH_INFO: `/v3/projects/8cd12e80bb0547548d34e65271e4e81e/users/389e0b98ea344f18bd28b27d5f57d6c7/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.100481 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd12e80bb0547548d34e65271e4e81e, user_id=389e0b98ea344f18bd28b27d5f57d6c7, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.103094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.120708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1347/2716] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:11 2022] PUT /identity/v3/projects/8cd12e80bb0547548d34e65271e4e81e/users/389e0b98ea344f18bd28b27d5f57d6c7/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:11.131345 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.131734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.132109 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.159607 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] MFA Rules not processed for user `389e0b98ea344f18bd28b27d5f57d6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:11.160926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:11.166560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:11.246070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1370/2717] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:11.250833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.299220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:11.332718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d9055-50bc-44f6-b791-7e052c2143b5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.337699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.338399 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.338686 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.338968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.339834 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.342148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.375951 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1348/2718] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:11 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:11.377136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.389138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92319e-49b1-47f5-bad7-5429e555484b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.466012 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.467311 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.467605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.467837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.468606 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.470657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.500062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1371/2719] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:11 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:11.501638 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.514126 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b385fb-a07c-468d-a9fc-7bb58f8b46ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.526670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.527673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.528114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.528544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.529471 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.531721 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.544181 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1349/2720] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:11.549429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.566695 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d836a2-ac4a-4001-a683-a5a8cd3431b6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.572635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.573713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.574005 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.574286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] PATH_INFO: `/v3/projects/9189b7de090c4b5889cb08770ee2d659/users/0813f73eb4ea466fb18e42a5b4c259fc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.597140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9189b7de090c4b5889cb08770ee2d659, user_id=0813f73eb4ea466fb18e42a5b4c259fc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.601138 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.627277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1372/2721] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:11 2022] PUT /identity/v3/projects/9189b7de090c4b5889cb08770ee2d659/users/0813f73eb4ea466fb18e42a5b4c259fc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:11.628303 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.653269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e93ec-ef78-41f9-a481-42b0fc647ac1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.717569 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.718372 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.718660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.718935 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.719719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.721348 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5934bac-21df-4dc2-a62b-c581af147e93 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.732381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1350/2722] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:11.736449 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.749922 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c9f3b-e8a4-4b4d-94d1-9ed3b982745d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:11.756157 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:11.757083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.757378 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.757651 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] PATH_INFO: `/v3/projects/9189b7de090c4b5889cb08770ee2d659/users/0813f73eb4ea466fb18e42a5b4c259fc/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.759707 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9189b7de090c4b5889cb08770ee2d659, user_id=0813f73eb4ea466fb18e42a5b4c259fc, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:11.762567 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:11.774490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1373/2723] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:11 2022] PUT /identity/v3/projects/9189b7de090c4b5889cb08770ee2d659/users/0813f73eb4ea466fb18e42a5b4c259fc/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:11.785963 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:11.786260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:11.786554 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:11.818800 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] MFA Rules not processed for user `0813f73eb4ea466fb18e42a5b4c259fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:11.820224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:11.912099 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1351/2724] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:11.925111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:11.927464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:12.001771 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa7d43-93f1-4c86-bc62-d78359025c77 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:12.005516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:12.006201 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:12.006486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:12.006767 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:12.017110 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:12.018715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:12.029861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1374/2725] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:49:12.105646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:12.115976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635ec23-9357-4dec-a1e1-bc121732b753 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:12.119867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:12.120553 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:12.120840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:12.121233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:12.149836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:12.182321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:12.183722 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfdf0c-ba04-4f17-a652-eb97781eae57 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:12.193838 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1352/2726] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:12.486319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:12.490887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:12.563552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a0c45-0a10-4d64-a77f-e9ee9a5f4488 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:12.568213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:12.569351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:12.569703 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:12.570044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] PATH_INFO: `/v3/users/389e0b98ea344f18bd28b27d5f57d6c7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:12.571604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389e0b98ea344f18bd28b27d5f57d6c7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:12.573705 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:12.638197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389e0b98ea344f18bd28b27d5f57d6c7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:12.655796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389e0b98ea344f18bd28b27d5f57d6c7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:12.656758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389e0b98ea344f18bd28b27d5f57d6c7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:12.667267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389e0b98ea344f18bd28b27d5f57d6c7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:12.678620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389e0b98ea344f18bd28b27d5f57d6c7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:12.690319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1375/2727] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:12 2022] DELETE /identity/v3/users/389e0b98ea344f18bd28b27d5f57d6c7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:12.691934 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:12.692802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:12.693383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:12.715631 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:12.718051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:12.804050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1353/2728] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:49:12.816359 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:12.817965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:12.902971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d612ab-91d7-4d6e-a7d7-bf2097174091 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:12.909267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:12.910076 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:12.910680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:12.910833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:12.922163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:12.923765 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:12.934908 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1376/2729] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:13.229494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:13.231262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:13.244509 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb92e-70c1-425b-983c-7ef0fb699238 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:13.255293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:13.256332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:13.256668 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:13.257011 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] PATH_INFO: `/v3/projects/8cd12e80bb0547548d34e65271e4e81e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:13.258312 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd12e80bb0547548d34e65271e4e81e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:13.260461 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:13.313407 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd12e80bb0547548d34e65271e4e81e', 'request_id': 'req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.321541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd12e80bb0547548d34e65271e4e81e', 'request_id': 'req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.328420 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd12e80bb0547548d34e65271e4e81e', 'request_id': 'req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.340766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] The token cache is being invalidate because project 8cd12e80bb0547548d34e65271e4e81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:13.341058 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.342179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.343801 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1354/2730] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:13 2022] DELETE /identity/v3/projects/8cd12e80bb0547548d34e65271e4e81e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:13.348362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:13.359290 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1eadc-8a77-42b5-9400-f17dc0cec87e service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:13.382820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:13.422127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:13.423026 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:13.423341 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:13.423620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] PATH_INFO: `/v3/users/0813f73eb4ea466fb18e42a5b4c259fc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:13.424832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0813f73eb4ea466fb18e42a5b4c259fc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:13.426779 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:13.476573 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0813f73eb4ea466fb18e42a5b4c259fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.489716 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0813f73eb4ea466fb18e42a5b4c259fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.490442 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0813f73eb4ea466fb18e42a5b4c259fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.499414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0813f73eb4ea466fb18e42a5b4c259fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.506670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b15be630-3d48-430c-bb1b-bb3c934308ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0813f73eb4ea466fb18e42a5b4c259fc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:13.512861 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1377/2731] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:13 2022] DELETE /identity/v3/users/0813f73eb4ea466fb18e42a5b4c259fc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:13.878767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:13.880296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:13.897596 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527ad27-337a-4c50-aaa1-a1e6e39b7d85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:13.979701 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:13.995384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:13.996311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:13.996602 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:13.997254 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] PATH_INFO: `/v3/projects/9189b7de090c4b5889cb08770ee2d659` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:13.998505 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9189b7de090c4b5889cb08770ee2d659)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.000606 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.090824 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9189b7de090c4b5889cb08770ee2d659', 'request_id': 'req-3c53e723-d580-4ed9-965a-d2a6c8982df6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.105902 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9189b7de090c4b5889cb08770ee2d659', 'request_id': 'req-3c53e723-d580-4ed9-965a-d2a6c8982df6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.114233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9189b7de090c4b5889cb08770ee2d659', 'request_id': 'req-3c53e723-d580-4ed9-965a-d2a6c8982df6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.149945 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] The token cache is being invalidate because project 9189b7de090c4b5889cb08770ee2d659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:14.150267 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.150949 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3c53e723-d580-4ed9-965a-d2a6c8982df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.153106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1355/2732] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:13 2022] DELETE /identity/v3/projects/9189b7de090c4b5889cb08770ee2d659 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:14.159491 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.160131 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.160702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.193140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:14.194824 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.279503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1378/2733] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:14.285275 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.285815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.285948 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.307436 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:14.308965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.320308 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1356/2734] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:14.322094 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.332766 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd63e1e-0fc5-46cc-a21f-5ea78e1ff943 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.348542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.349251 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.349533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.349810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.350563 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:49:14.350876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.352583 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.370306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1379/2735] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:49:14.371913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.386247 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f51a-94bd-4591-823a-e9b741ae5521 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.399331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.400333 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.400784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.401670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.402379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.404604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.438120 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1357/2736] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:14 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:14.442271 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.453794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b73b7-06a1-4e32-b0d5-7e896228af60 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.486070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:14.516171 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.516782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.517063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.517368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.518044 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.520052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.542008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1380/2737] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:14 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:14.546462 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.557893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc653e-0221-482e-ab9c-6f1998855d2c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.563260 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.564022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.564315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.564623 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.565362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.567004 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.579098 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1358/2738] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:14.585876 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.597170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9ad1c-34ed-4016-8898-1f3a47980c31 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.602928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.603681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.603963 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.604260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.605000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.606605 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.620756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1381/2739] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:14.621968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.632301 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806adfe-2155-4729-b5af-813edc41e317 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.637866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.639173 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.639532 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.639910 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] PATH_INFO: `/v3/projects/b00c3652b99840009666b9f12be0a69b/users/6552db969a6643f9b79fa65cb6c6beb4/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.666237 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00c3652b99840009666b9f12be0a69b, user_id=6552db969a6643f9b79fa65cb6c6beb4, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.669010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150d47e-5519-4412-914d-821d48d9d0f9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.684087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1359/2740] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:14 2022] PUT /identity/v3/projects/b00c3652b99840009666b9f12be0a69b/users/6552db969a6643f9b79fa65cb6c6beb4/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:14.685826 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.687756 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.697171 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.697513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.697723 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.702196 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc5784-c607-440b-a0b8-54d597cb81ed admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.726179 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] MFA Rules not processed for user `6552db969a6643f9b79fa65cb6c6beb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:14.727434 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.765839 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.766809 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.767033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.767351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] PATH_INFO: `/v3/users/544cf118cc9e4aed8e7ec7e9dfe023bc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.768590 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] RBAC: Authorizing `identity:delete_user(user_id=544cf118cc9e4aed8e7ec7e9dfe023bc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.770553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:14.798921 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1360/2741] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:14.810774 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:14.812392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.821859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:14.824386 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544cf118cc9e4aed8e7ec7e9dfe023bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.837952 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544cf118cc9e4aed8e7ec7e9dfe023bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.838813 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544cf118cc9e4aed8e7ec7e9dfe023bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.847719 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544cf118cc9e4aed8e7ec7e9dfe023bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.855133 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544cf118cc9e4aed8e7ec7e9dfe023bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:14.861627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1382/2742] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:14 2022] DELETE /identity/v3/users/544cf118cc9e4aed8e7ec7e9dfe023bc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:14.875410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629a62-237f-4a90-89d1-df240d1fca9e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:14.924898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:14.925565 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:14.925850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:14.926131 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:14.928100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:14.991685 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:14.993205 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89830745-42f8-47e5-aeb7-4765bf2b5be3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.004000 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1361/2743] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:49:15.048406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.050046 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:15.090483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:15.126141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d78b-4b7b-43fa-8b6a-20557757585d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.130796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.131715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.132003 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.132283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] PATH_INFO: `/v3/projects/f64a8807499a44ababb8bb037fc3d195` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.133498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64a8807499a44ababb8bb037fc3d195)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.135490 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.196286 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64a8807499a44ababb8bb037fc3d195', 'request_id': 'req-dd289171-5658-4ef2-951b-84ec1d2425b9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:15.203219 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64a8807499a44ababb8bb037fc3d195', 'request_id': 'req-dd289171-5658-4ef2-951b-84ec1d2425b9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:15.211777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64a8807499a44ababb8bb037fc3d195', 'request_id': 'req-dd289171-5658-4ef2-951b-84ec1d2425b9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:15.224416 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] The token cache is being invalidate because project f64a8807499a44ababb8bb037fc3d195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:15.224773 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:15.225556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dd289171-5658-4ef2-951b-84ec1d2425b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:15.227453 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1383/2744] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:15 2022] DELETE /identity/v3/projects/f64a8807499a44ababb8bb037fc3d195 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:15.233773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.235331 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.235913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.258807 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:15.260178 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:15.351249 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1362/2745] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:15.359880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.360742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.361030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.381729 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:15.382997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:15.391688 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1384/2746] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:49:15.396044 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.406030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6679a-4a66-4231-9669-fdba8f5fc363 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.410597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.411306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.411580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.411860 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.412590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:49:15.412901 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.414519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.426289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1363/2747] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:49:15.430736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.440525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae147b-48e9-4749-abae-f845357fc401 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.446030 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.446803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.447126 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.447427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.448117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.450092 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.482515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1385/2748] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:15 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:15.487221 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.493813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:15.497752 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979ffce-0e89-4eb3-b415-ef69f9a9a4c1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.561334 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.562148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.562498 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.562883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.563767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.565960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.590105 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1364/2749] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:15 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:15.595137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.605547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5706fb-5f54-4310-9721-2dfe4b77d55d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.610350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.611023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.611343 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.611625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.612362 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.613995 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.625197 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1386/2750] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:15.629269 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.640413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2609-95b9-4e13-9634-3e3b742a87ae admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.645884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.646672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.646952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.647344 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.648075 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.649710 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb032b-4cd8-4362-be8d-db3d66fbcbb4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.660516 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1365/2751] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:15.665059 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.676385 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da702-5f68-4b17-884c-0c226e21e564 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.681872 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.682784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.683111 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.683411 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] PATH_INFO: `/v3/projects/9c90246f01ad410ea9569c51eb5f2ecb/users/3477f21462c84759b0762cf13427b3c8/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.701288 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c90246f01ad410ea9569c51eb5f2ecb, user_id=3477f21462c84759b0762cf13427b3c8, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.704458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.721417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1387/2752] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:15 2022] PUT /identity/v3/projects/9c90246f01ad410ea9569c51eb5f2ecb/users/3477f21462c84759b0762cf13427b3c8/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:49:15.728555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.728850 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.729176 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.753357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:15.755024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:15.823731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1366/2753] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:15.833675 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:15.835682 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:15.894102 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c4093-e0d6-46e7-b984-069d8f339af6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:15.898127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:15.898784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:15.899088 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:15.899381 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:15.909113 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:15.911156 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1ed7-63a2-43e2-b5d4-fb48d1698f84 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:15.915101 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:15.921086 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1388/2754] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:17.619919 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:17.621566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:17.639814 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536262ef-61f6-45b0-b0fc-4b9c35527892 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:17.646522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:17.647761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:17.648043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:17.648425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] PATH_INFO: `/v3/users/6552db969a6643f9b79fa65cb6c6beb4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:17.649697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6552db969a6643f9b79fa65cb6c6beb4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:17.653263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:17.728725 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6552db969a6643f9b79fa65cb6c6beb4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:17.752506 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6552db969a6643f9b79fa65cb6c6beb4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:17.770512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6552db969a6643f9b79fa65cb6c6beb4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:17.782360 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6552db969a6643f9b79fa65cb6c6beb4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:17.783210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6552db969a6643f9b79fa65cb6c6beb4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:17.800604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:17.811976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1367/2755] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:17 2022] DELETE /identity/v3/users/6552db969a6643f9b79fa65cb6c6beb4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:17.818432 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:17.819919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:17.820233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:17.841864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:17.843422 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:17.913161 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1389/2756] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:17.924514 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:17.931966 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:17.998649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075d61a-d63b-484c-ab18-156cfca2c7ca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.002873 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.003586 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.003884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.004148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.016210 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.018305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ecec4-b271-4622-8d61-ddd238045a48 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:18.030342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1368/2757] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:18.374575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:18.375690 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:18.399489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a26c4-805b-41c4-b258-33166e66e3b2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.404225 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.405355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.405641 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.405917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] PATH_INFO: `/v3/projects/b00c3652b99840009666b9f12be0a69b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.407168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00c3652b99840009666b9f12be0a69b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.409097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:18.468496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00c3652b99840009666b9f12be0a69b', 'request_id': 'req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:18.476176 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00c3652b99840009666b9f12be0a69b', 'request_id': 'req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:18.487481 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00c3652b99840009666b9f12be0a69b', 'request_id': 'req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:18.502700 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] The token cache is being invalidate because project b00c3652b99840009666b9f12be0a69b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:18.502923 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:18.503582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-94b4fed4-6e66-46f2-9fbc-b86a34f1ba51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:18.505397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1390/2758] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:18 2022] DELETE /identity/v3/projects/b00c3652b99840009666b9f12be0a69b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:18.510332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.510628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.510913 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.532815 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:18.534537 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:18.620167 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1369/2759] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:18.630658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.631069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.631868 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.656796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:18.658346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:18.683213 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1391/2760] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:18.687842 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:18.700225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a1c6e-753a-4a0f-8f92-21f600b3f1ff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.706111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.707274 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.707641 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.708714 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.711220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:49:18.711220 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.713094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:18.728871 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1370/2761] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:49:18.733041 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:18.748640 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9063c-3a08-44dc-8f8b-ccfd7a620624 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.756229 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.756931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.757323 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.757611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.758500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.762523 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:18.813117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1392/2762] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:18 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:18.816908 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:18.829072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2468fa-9746-4252-bb8a-e8a0ac02901d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.916731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.917441 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.917735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.918014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.918724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.921266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:18.949380 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1371/2763] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:18 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:18.953821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:18.970476 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc03b-6fd1-4080-b1c8-d7e7d364c4e1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:18.978205 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:18.979314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:18.979832 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:18.980337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:18.981337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:18.984051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.001025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1393/2764] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:19.003390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:19.015885 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d22a2-583d-44b3-b72d-2b6fac78f660 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:19.021035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:19.021884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.022253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.022784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.023702 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:19.026104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7659-0b84-4de3-aac5-9d17ae80b9f8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.043517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1372/2765] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:19.045757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:19.061772 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad713c-24b2-4cec-80ff-8397c212f6ac admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:19.068680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:19.069036 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.069437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.069815 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] PATH_INFO: `/v3/projects/aad0a716fd7940729cacb72af827d97c/users/dbf190d492a749ffa60743e97d351a5d/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.087960 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad0a716fd7940729cacb72af827d97c, user_id=dbf190d492a749ffa60743e97d351a5d, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:19.090732 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.109180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1394/2766] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:19 2022] PUT /identity/v3/projects/aad0a716fd7940729cacb72af827d97c/users/dbf190d492a749ffa60743e97d351a5d/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:19.120447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.121760 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.122149 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.131435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:19.155840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] MFA Rules not processed for user `dbf190d492a749ffa60743e97d351a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:19.157288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:19.285022 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1373/2767] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:19.306056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:19.309459 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:19.348320 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:19.408966 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298aa73-e689-439d-b323-02f7ee848e3a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:19.413396 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:19.414095 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.414527 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.416051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.429681 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:19.431704 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c11d-cea5-46ca-b4ca-e8d561304a0e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.446377 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1395/2768] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:19.490217 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:19.607174 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9b584-5986-4fef-b37c-c451e7c19d32 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:19.613124 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:19.614088 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.614377 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.614655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] PATH_INFO: `/v3/users/dbf190d492a749ffa60743e97d351a5d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.616078 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf190d492a749ffa60743e97d351a5d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:19.618200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.670750 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf190d492a749ffa60743e97d351a5d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:19.680161 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf190d492a749ffa60743e97d351a5d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:19.688933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf190d492a749ffa60743e97d351a5d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:19.689953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf190d492a749ffa60743e97d351a5d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:19.690877 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf190d492a749ffa60743e97d351a5d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:19.711032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1374/2769] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:49:19 2022] DELETE /identity/v3/users/dbf190d492a749ffa60743e97d351a5d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:19.716710 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1aec-c24b-4672-a7e0-3199f354e146 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.717006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1aec-c24b-4672-a7e0-3199f354e146 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.717299 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f1aec-c24b-4672-a7e0-3199f354e146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.736770 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-195f1aec-c24b-4672-a7e0-3199f354e146 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:19.738150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-195f1aec-c24b-4672-a7e0-3199f354e146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:19.827502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1396/2770] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:19.838805 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:19.840716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:19.858475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:19.929090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c33c4e-19cf-4dbc-8741-a796a6fad1e7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:19.932604 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:19.933412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:19.933794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:19.934165 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:19.945297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:19.946907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:19.963101 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1375/2771] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:49:20.252984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.254023 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.254650 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.290782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] MFA Rules not processed for user `3477f21462c84759b0762cf13427b3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:20.292618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.376314 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:20.377851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.397392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a073b4-2f90-4e97-bbc1-f7ad7779238d service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:20.397669 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1397/2772] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:20.404538 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:20.405952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.406304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.406683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] PATH_INFO: `/v3/projects/aad0a716fd7940729cacb72af827d97c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.408117 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad0a716fd7940729cacb72af827d97c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:20.410207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:20.413423 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:20.415209 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.469382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad0a716fd7940729cacb72af827d97c', 'request_id': 'req-c60d6eee-dc78-48f8-b83c-fb89acac5904'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:20.479421 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad0a716fd7940729cacb72af827d97c', 'request_id': 'req-c60d6eee-dc78-48f8-b83c-fb89acac5904'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:20.480318 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:20.487114 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad0a716fd7940729cacb72af827d97c', 'request_id': 'req-c60d6eee-dc78-48f8-b83c-fb89acac5904'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:20.502761 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] The token cache is being invalidate because project aad0a716fd7940729cacb72af827d97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:49:20.503061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:20.503947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c60d6eee-dc78-48f8-b83c-fb89acac5904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:49:20.505784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1376/2773] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:49:20 2022] DELETE /identity/v3/projects/aad0a716fd7940729cacb72af827d97c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:49:20.512264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e195b-6d14-4c80-b470-501e681d9f9e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.513555 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e195b-6d14-4c80-b470-501e681d9f9e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.513926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e195b-6d14-4c80-b470-501e681d9f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.515753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375cfae-2b84-4457-a5fe-bcc81cd04722 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:20.540104 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b81e195b-6d14-4c80-b470-501e681d9f9e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:20.542258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b81e195b-6d14-4c80-b470-501e681d9f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.627284 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:20.628522 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.628977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.629503 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.631111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1377/2774] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:20.636435 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.640409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.640708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.640990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.662632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:20.663888 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:20.672839 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1378/2775] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:20.679108 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:20.692598 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70058c72-4676-4e4d-a9a0-a9976b834b82 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:20.698266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:20.699315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.699891 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.700463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.701639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:49:20.702392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:20.704543 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:20.726475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1379/2776] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:49:20.726475 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:20.736336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:20.738968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:20.739319 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16d68a-03f3-499f-80eb-48eb935023af admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:20.754315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:20.755008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.755326 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.755973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.756322 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:20.761390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:20.761620 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1398/2777] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:49:20.790833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:20.807197 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1380/2778] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:20 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:20.809127 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:20.829468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cb1bb-4e56-487b-b5b0-087de82a3f5d service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:20.960017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:20.960990 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:20.961267 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:20.961912 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:20.962566 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:20.966125 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:21.001862 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:21.011873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1399/2779] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:20 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:49:21.014091 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850da34-5785-465e-8054-8ac0a103804d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:21.019691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:21.020405 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:21.020696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:21.021074 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:21.022235 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:21.024121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:21.040593 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:21.041066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1381/2780] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:21.061192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7346bd-daa7-40b8-8dfc-51f8923c994a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:21.071534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:21.072915 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:21.074027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:21.074746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:21.083184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:21.090055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf328f3e-9a8e-4037-b522-1c4090f7c98b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:21.103028 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1400/2781] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:21.107435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:21.123624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17557cf-0d4b-40fb-84af-f9b40681271a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:21.130266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:21.131233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:21.131520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:21.131804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] PATH_INFO: `/v3/projects/47cfee44c4264b0a8cfb5a93b7f5378a/users/b2c58a25e4cc44a8b1fd9e1edfabdc02/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:21.150608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47cfee44c4264b0a8cfb5a93b7f5378a, user_id=b2c58a25e4cc44a8b1fd9e1edfabdc02, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:21.153734 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:21.172063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1382/2782] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:21 2022] PUT /identity/v3/projects/47cfee44c4264b0a8cfb5a93b7f5378a/users/b2c58a25e4cc44a8b1fd9e1edfabdc02/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:49:21.180183 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b8e57-64b8-4bb2-95bc-4f7cea567a37 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:21.180502 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b8e57-64b8-4bb2-95bc-4f7cea567a37 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:21.180781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b8e57-64b8-4bb2-95bc-4f7cea567a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:21.190584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:21.204062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-928b8e57-64b8-4bb2-95bc-4f7cea567a37 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:21.205426 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-928b8e57-64b8-4bb2-95bc-4f7cea567a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:21.283687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1401/2783] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:49:21.297848 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:21.299695 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:21.380207 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acfba9-1284-4225-9779-68ecc1e46d7e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:21.389854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:21.390976 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:21.391357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:21.391660 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:21.402691 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:21.404544 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:21.417535 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1383/2784] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:22.796052 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260595e-1ced-41f4-9305-ea22978c0dc6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:22.796542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260595e-1ced-41f4-9305-ea22978c0dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:22.796643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260595e-1ced-41f4-9305-ea22978c0dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:22.827837 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0260595e-1ced-41f4-9305-ea22978c0dc6 None None] MFA Rules not processed for user `3477f21462c84759b0762cf13427b3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:22.829390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0260595e-1ced-41f4-9305-ea22978c0dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:22.925608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1402/2785] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:24.538424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:24.550153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd190a-f0cf-4d99-887c-7c1df59b091b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:24.555146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:24.556062 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:24.556519 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:24.556952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:24.561990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:24.725508 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:24.727351 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:24.771297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1384/2786] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:49:30.148017 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.149191 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.149515 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.183755 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] MFA Rules not processed for user `b2c58a25e4cc44a8b1fd9e1edfabdc02`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:30.185185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:30.277467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1403/2787] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:30.281432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.293467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d0c14-1c2a-475b-a71e-7a472bdf6287 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.298628 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.299289 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.299580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.299869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.300583 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.302529 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.343141 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1385/2788] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:49:30 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:30.344982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.355887 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6f65-9277-476c-a3e5-b19ef9cb6341 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.395417 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:30.415929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.416752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.417033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.417305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.417992 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.420039 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.446500 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1404/2789] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:49:30 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:49:30.447960 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.460031 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc3a1c-d00d-463a-8b44-2210d23d1178 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.465678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.466640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.466933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.467234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.467977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.469616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.481806 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1386/2790] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:30.486379 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.496972 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e691-0be0-4da3-b8dc-bdf15686c1fa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.503033 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.503989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.504273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.504552 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] PATH_INFO: `/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3/users/3677d760764d48e9be49b5bfa1fa8ba6/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.521743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab460c4e3e046fea8888e9c1bf79cb3, user_id=3677d760764d48e9be49b5bfa1fa8ba6, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.524470 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.538192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1405/2791] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:30 2022] PUT /identity/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3/users/3677d760764d48e9be49b5bfa1fa8ba6/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:49:30.542191 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.553626 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5a1b-17e8-4d58-a4d7-4481301c24eb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.579547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:30.618255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.618974 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.619295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.619574 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.620310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.621971 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a2460-c3f1-4df0-a831-a08e3096d4d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.632973 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1387/2792] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:49:30.637548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:30.651055 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07bb7-9631-45ab-aac7-b6faf14c93fe admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:30.656140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:30.657047 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:30.657331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:30.657609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] PATH_INFO: `/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3/users/3677d760764d48e9be49b5bfa1fa8ba6/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:30.659601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab460c4e3e046fea8888e9c1bf79cb3, user_id=3677d760764d48e9be49b5bfa1fa8ba6, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:30.662279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:30.673635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1406/2793] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:49:30 2022] PUT /identity/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3/users/3677d760764d48e9be49b5bfa1fa8ba6/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:49:35.356645 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bcff6-282d-4bc9-bed4-f87de0f73bde None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:35.357272 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bcff6-282d-4bc9-bed4-f87de0f73bde None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:35.357433 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bcff6-282d-4bc9-bed4-f87de0f73bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:35.388051 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-400bcff6-282d-4bc9-bed4-f87de0f73bde None None] MFA Rules not processed for user `3677d760764d48e9be49b5bfa1fa8ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:35.389305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-400bcff6-282d-4bc9-bed4-f87de0f73bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:35.473915 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1388/2794] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:35.483349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:49:35.485421 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:35.570263 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc97e9-3e7d-4c40-9272-bbb2816fd423 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:49:35.576292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:49:35.576965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:35.577252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:35.577533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:35.659737 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:49:35.661547 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:49:35.671486 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1407/2795] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:49:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:49:36.859836 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:49:36.860394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:49:36.860394 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:49:36.878947 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] MFA Rules not processed for user `b2c58a25e4cc44a8b1fd9e1edfabdc02`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:49:36.880385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:49:36.887887 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1389/2796] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:49:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:49:45.666797 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:49:46.873296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=54 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:08.614803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:08.673672 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398aa30-12d1-4576-9d6b-ebe4110f030e service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:08.677882 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:08.678546 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:08.678827 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:08.679104 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:08.689534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:08.690977 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca38222-cf50-432b-a91b-41cca591021a service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:08.701200 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1408/2797] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:50:18.696427 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=26 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:50:52.686784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:50:52.688912 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:52.768933 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710bab-eaa1-4ebc-ab7b-c6c8326f6517 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:50:52.774383 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:50:52.775042 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:50:52.775354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:50:52.775632 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:50:52.777370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:50:52.789699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:50:52.791385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:50:52.801817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1390/2798] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:50:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:51:02.797478 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=32 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:51:27.899551 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:51:27.900100 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:51:27.900247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:51:27.919270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] MFA Rules not processed for user `565cc9705a94454b9e68595b85c6a286`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:51:27.920519 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:51:27.987404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1409/2799] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:51:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:51:37.979997 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=25 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:01.101724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:01.112982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a895854-2b48-4809-9766-aa78f024496e service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:01.116828 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:01.117486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:01.117773 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:01.118045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:01.120123 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:01.135264 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:01.136697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e22c8e-42d4-4dc9-a70b-46ae2a832b7d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:01.148492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1391/2800] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:02.338733 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:02.349467 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb860425-8294-4c0e-bee6-c26cf606572b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:02.353751 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:02.354413 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:02.354764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:02.355042 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:02.357136 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:02.372252 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:02.373711 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:02.384970 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1410/2801] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:52:06.730102 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:06.730609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:06.730708 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:06.748709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] MFA Rules not processed for user `3477f21462c84759b0762cf13427b3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:52:06.749961 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:06.816258 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1392/2802] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:52:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:52:12.379867 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:16.808809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=31 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:52:28.916168 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:52:28.928628 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1c427-abab-4cc9-8330-a83f695ab96d service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:52:28.933137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:52:28.933794 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:52:28.934081 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:52:28.934355 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:52:28.936602 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:52:28.947764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:52:28.949223 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:52:28.958053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1411/2803] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:52:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:52:38.953931 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:18.039063 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:18.040735 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:18.064448 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340b31a-b071-4275-8d5b-e6e042250d48 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:18.071584 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:18.072368 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:18.072666 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:18.072942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:18.074939 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:18.093983 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:18.095659 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:18.111035 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1393/2804] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:26.755622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:26.757296 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:26.773957 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001f25f-e274-4b61-93ba-2919202eb3bc service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:26.782188 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:26.782965 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:26.783429 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:26.783782 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:26.786559 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:26.809528 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:26.811658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:26.827144 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1412/2805] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:26.839087 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:26.840762 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:26.926840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230832d-c171-486e-b739-0eccca3005c1 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:26.931957 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:26.932866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:26.933150 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:26.933541 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] PATH_INFO: `/v3/users/3477f21462c84759b0762cf13427b3c8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:26.934817 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3477f21462c84759b0762cf13427b3c8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:26.936823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:26.989704 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3477f21462c84759b0762cf13427b3c8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:26.997164 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3477f21462c84759b0762cf13427b3c8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.004895 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3477f21462c84759b0762cf13427b3c8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.005517 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3477f21462c84759b0762cf13427b3c8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.006130 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5c19a65f-d958-4a23-ad7b-571d62489166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3477f21462c84759b0762cf13427b3c8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.024522 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1394/2806] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:26 2022] DELETE /identity/v3/users/3477f21462c84759b0762cf13427b3c8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:27.221889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.233631 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c56c66-262d-48d0-ab27-15af6a9e5e4f service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.320063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.321021 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.321314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.321591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] PATH_INFO: `/v3/projects/9c90246f01ad410ea9569c51eb5f2ecb` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.322801 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c90246f01ad410ea9569c51eb5f2ecb)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.324962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.376811 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c90246f01ad410ea9569c51eb5f2ecb', 'request_id': 'req-05bafdda-ceae-473b-9be5-34bf4360ebef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.383121 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c90246f01ad410ea9569c51eb5f2ecb', 'request_id': 'req-05bafdda-ceae-473b-9be5-34bf4360ebef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.391928 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c90246f01ad410ea9569c51eb5f2ecb', 'request_id': 'req-05bafdda-ceae-473b-9be5-34bf4360ebef'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.403542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] The token cache is being invalidate because project 9c90246f01ad410ea9569c51eb5f2ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:27.403542 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.412448 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-05bafdda-ceae-473b-9be5-34bf4360ebef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:27.414155 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1413/2807] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:27 2022] DELETE /identity/v3/projects/9c90246f01ad410ea9569c51eb5f2ecb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:27.420381 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.420650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.420952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.440646 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:27.441869 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:27.519050 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1395/2808] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:53:27.527368 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.527638 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.527919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.547525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:27.548764 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:27.558905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1414/2809] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:27.563794 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.577295 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99481027-aa5d-4e74-9571-b27c5a1ce3e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.583706 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.584389 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.584677 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.584938 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.585671 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:53:27.585977 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.587648 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.589369 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:27.609719 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1396/2810] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:53:27.611339 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.622799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7f616-8393-4c69-b84d-f941b61a11e1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.628314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.629008 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.629292 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.629574 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.630279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.632270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.669885 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1415/2811] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:27 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:53:27.670658 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.687224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81289f-07f5-4cde-86af-b351ede1ade6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.758371 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.759112 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.759398 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.759680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.760384 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.763425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.786129 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1397/2812] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:27 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:53:27.790497 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.808985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf215-cff9-45ec-a800-db830b7b8cb1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.814006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.814722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.815006 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.815313 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.816056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.817721 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.829524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1416/2813] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:27.834413 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.846135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3e0e7-dfd7-41a3-9283-e395b6941fc6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.851281 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.851965 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.852252 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.852526 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.853332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.857231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136c97-3351-412f-b019-049cb48b248f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.868261 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1398/2814] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:27.873728 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:27.884465 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047109dd-6889-4a58-a6af-18300d4bd875 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:27.891784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:27.892781 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.893060 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.893332 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] PATH_INFO: `/v3/projects/749ebb5f332f43678afdf4f7af7012d1/users/1a3c52c97f3a448585fde2cb5bf5041b/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.910857 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749ebb5f332f43678afdf4f7af7012d1, user_id=1a3c52c97f3a448585fde2cb5bf5041b, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:27.913648 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:27.929141 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1417/2815] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:27 2022] PUT /identity/v3/projects/749ebb5f332f43678afdf4f7af7012d1/users/1a3c52c97f3a448585fde2cb5bf5041b/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:53:27.940026 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:27.940299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:27.940589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:27.974473 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] MFA Rules not processed for user `1a3c52c97f3a448585fde2cb5bf5041b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:27.975898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:28.061234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1399/2816] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:28.072652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:28.075406 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:28.092071 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:28.165093 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e12-1b76-42fe-9ec2-15d214205a15 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:28.172105 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:28.172787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:28.173069 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:28.173342 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:28.182336 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:28.184582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ed44e-209e-4225-86cf-532f631e4b17 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:28.194752 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1418/2817] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:28.432364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:28.505876 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271dd0-d68b-47bb-a725-10b9916af780 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:28.512822 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:28.514002 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:28.514373 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:28.514763 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] PATH_INFO: `/v3/users/1a3c52c97f3a448585fde2cb5bf5041b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:28.516696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3c52c97f3a448585fde2cb5bf5041b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:28.519187 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:28.521970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:28.574984 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3c52c97f3a448585fde2cb5bf5041b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:28.582014 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3c52c97f3a448585fde2cb5bf5041b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:28.596903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3c52c97f3a448585fde2cb5bf5041b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:28.596903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3c52c97f3a448585fde2cb5bf5041b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:28.596903 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3c52c97f3a448585fde2cb5bf5041b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:28.614591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1400/2818] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:28 2022] DELETE /identity/v3/users/1a3c52c97f3a448585fde2cb5bf5041b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:53:28.619437 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:28.619734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:28.620019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:28.639029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:28.640810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:28.708916 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1419/2819] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:53:28.719754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:28.721311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:28.789304 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fac5df-6963-4226-b507-a342dea98906 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:28.793637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:28.794379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:28.794667 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:28.794958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:28.805867 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:28.807370 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598872e-50df-46ab-8a55-96edec7e0663 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:28.817841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1401/2820] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:29.121057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.122753 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:29.136984 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c642a-2495-4422-b6c7-fdce8ec10516 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.143123 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.144040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.144324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.144601 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] PATH_INFO: `/v3/projects/749ebb5f332f43678afdf4f7af7012d1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.145985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749ebb5f332f43678afdf4f7af7012d1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.148020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.204610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749ebb5f332f43678afdf4f7af7012d1', 'request_id': 'req-76c100bf-774a-4155-bfd6-bbc1fb632275'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:29.213657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749ebb5f332f43678afdf4f7af7012d1', 'request_id': 'req-76c100bf-774a-4155-bfd6-bbc1fb632275'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:29.222810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749ebb5f332f43678afdf4f7af7012d1', 'request_id': 'req-76c100bf-774a-4155-bfd6-bbc1fb632275'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:29.233877 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] The token cache is being invalidate because project 749ebb5f332f43678afdf4f7af7012d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:29.234080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:29.234726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-76c100bf-774a-4155-bfd6-bbc1fb632275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:29.236623 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1420/2821] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:29 2022] DELETE /identity/v3/projects/749ebb5f332f43678afdf4f7af7012d1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:29.241678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.242454 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.242550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.266552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:29.267906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:29.335620 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1402/2822] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:29.344673 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.344920 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.345199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.364944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:29.366164 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:29.375636 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1421/2823] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:29.380224 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.391630 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ad07-d42b-4e96-a86a-afb1d3adf8bc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.398534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.399556 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.399970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.400346 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.401520 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:53:29.402018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.404233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.416943 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1403/2824] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:53:29.421722 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.431715 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3fe4-ebad-4464-8667-96ba6c35b528 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.437584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.438291 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.438558 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.438846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.439591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.441548 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.456787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:29.476556 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1422/2825] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:29 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:29.481785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.492403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069d9c6-5357-4ef9-8276-cfdd819cf084 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.501081 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:29.553697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.554386 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.554672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.554946 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.555683 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.557635 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.580493 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1404/2826] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:29 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:53:29.585268 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.596532 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834110c8-3da0-4676-bc54-0ee9356190cc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.602035 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.602653 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.602934 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.603232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.604029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.605647 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.617953 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1423/2827] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:29.622696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.636496 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2451b-4096-4eb7-89b4-ce5ef77019dd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.642592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.643354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.643640 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.643916 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.644654 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.646283 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986d22b-4a46-476b-bb55-e0db609a78e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.657724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1405/2828] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:29.662632 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.673987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46076-b553-4eb7-9f6d-dd2db462da77 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.679001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.679933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.680217 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.680496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] PATH_INFO: `/v3/projects/af84292963364f1c8c0c8cc5d7316f2c/users/2fe3793a6e9b43caa825fe3d111edc77/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.697163 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af84292963364f1c8c0c8cc5d7316f2c, user_id=2fe3793a6e9b43caa825fe3d111edc77, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.700137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.719020 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1424/2829] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:29 2022] PUT /identity/v3/projects/af84292963364f1c8c0c8cc5d7316f2c/users/2fe3793a6e9b43caa825fe3d111edc77/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:53:29.732492 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.732796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.733073 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.767990 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] MFA Rules not processed for user `2fe3793a6e9b43caa825fe3d111edc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:29.769253 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:29.842767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1406/2830] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:29.877525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:29.879458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:29.945177 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2bcb7-66dd-4e0e-ba03-75b5b8ad4021 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:29.949557 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:29.950968 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:29.951265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:29.951545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:29.964816 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:29.967814 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb6380-a6bf-4684-ba85-7f5d975d3986 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:29.977433 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1425/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:29.991925 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:30.059597 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccadac-5067-450e-aed1-d5021cfa6705 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:30.065497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:30.066446 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:30.066727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:30.067028 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] PATH_INFO: `/v3/users/2fe3793a6e9b43caa825fe3d111edc77` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:30.068412 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe3793a6e9b43caa825fe3d111edc77)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:30.070400 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:30.094813 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:30.131558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe3793a6e9b43caa825fe3d111edc77'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:30.139350 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe3793a6e9b43caa825fe3d111edc77'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:30.148513 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe3793a6e9b43caa825fe3d111edc77'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:30.149196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe3793a6e9b43caa825fe3d111edc77'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:30.149809 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe3793a6e9b43caa825fe3d111edc77'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:30.169639 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1407/2832] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:29 2022] DELETE /identity/v3/users/2fe3793a6e9b43caa825fe3d111edc77 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:30.174866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:30.175473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:30.175473 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:30.199498 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:30.200820 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:30.271415 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1426/2833] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:30.281731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:30.283339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:30.352581 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88a41-3099-46aa-b9bd-42cebf8287c8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:30.358382 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:30.359067 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:30.359390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:30.359670 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:30.371121 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:30.372609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040d372-2e79-48e0-89fa-4e5c0706fc82 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:30.383297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1408/2834] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:30.939350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:30.941019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:30.957588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e62dd2-9aaf-4655-964f-a1969f48b70e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:30.963404 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:30.964319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:30.964603 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:30.964883 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] PATH_INFO: `/v3/projects/af84292963364f1c8c0c8cc5d7316f2c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:30.966810 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af84292963364f1c8c0c8cc5d7316f2c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:30.968944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.024580 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af84292963364f1c8c0c8cc5d7316f2c', 'request_id': 'req-1b30323d-2473-4042-a404-bd8f21956882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:31.031608 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af84292963364f1c8c0c8cc5d7316f2c', 'request_id': 'req-1b30323d-2473-4042-a404-bd8f21956882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:31.034380 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:31.040727 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af84292963364f1c8c0c8cc5d7316f2c', 'request_id': 'req-1b30323d-2473-4042-a404-bd8f21956882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:31.053260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] The token cache is being invalidate because project af84292963364f1c8c0c8cc5d7316f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:31.053543 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:31.054220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1b30323d-2473-4042-a404-bd8f21956882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:31.055838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1427/2835] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:30 2022] DELETE /identity/v3/projects/af84292963364f1c8c0c8cc5d7316f2c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:31.061329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.061572 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.061880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.086497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:31.087718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:31.157288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1409/2836] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:31.167364 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.167618 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.167918 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.189224 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:31.190369 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:31.199833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1428/2837] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:31.204552 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.215900 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f34d2-46a8-4e05-b922-334c945494bd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.221330 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.222135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.222592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.223037 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.224186 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:53:31.224521 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.226148 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.248416 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1410/2838] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:53:31.253407 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.265145 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcabea2-ff6b-4a79-b312-8aed4cffb9b4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.270787 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.271508 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.271777 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.272048 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.272742 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.274680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.304489 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1429/2839] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:31.309046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.319958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e98af-bb1d-433f-bc57-82050c24be17 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.346854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:31.380726 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.381418 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.381702 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.381968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.382653 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.384600 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.409985 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1411/2840] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:31.415613 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.427517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24b2d-a7ab-4912-9b5a-c0fa7253abce admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.433057 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.433749 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.434031 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.434337 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.435097 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.436865 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.450784 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1430/2841] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:31.452782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.464133 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c24781-c5e0-4725-ad38-863f6489bada admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.470715 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.471435 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.471718 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.471987 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.472728 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.474379 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62106335-bab2-4965-b998-c6728d032f73 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.485826 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1412/2842] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:31.490212 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.507279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f530-b025-4294-a7bc-1c974b7fff93 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.513293 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.514462 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.514998 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.515516 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] PATH_INFO: `/v3/projects/5491c649b1164ebf95ff8b8b7b289b87/users/3c102f284b0b4ff99b333c3af860bd70/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.534401 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5491c649b1164ebf95ff8b8b7b289b87, user_id=3c102f284b0b4ff99b333c3af860bd70, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.546748 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.571479 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1431/2843] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:31 2022] PUT /identity/v3/projects/5491c649b1164ebf95ff8b8b7b289b87/users/3c102f284b0b4ff99b333c3af860bd70/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:31.583305 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.583637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.583854 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.613851 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] MFA Rules not processed for user `3c102f284b0b4ff99b333c3af860bd70`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:31.615189 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:31.683942 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1413/2844] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:31.689170 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.764331 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cddbda-0018-40d1-ad00-caf38adf4b61 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.772664 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.773348 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.773660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.773926 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.774627 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.776634 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.806314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1432/2845] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:31.810676 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.822592 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce6beb-9d6f-40a4-bece-ebdfe31578af None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.895767 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.896463 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.896748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.897024 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.897716 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.900883 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.927030 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1414/2846] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:31 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:31.933475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.951344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da90d-df7e-4a7e-bff2-a1b1b875d230 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.956982 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.958034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.958319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.958594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.959609 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:31.961207 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:31.975848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1433/2847] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:31.980313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:31.991311 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ab23f-66d0-4bfe-985a-f49c37bfe092 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:31.996700 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:31.997338 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:31.997616 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:31.997889 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:31.998655 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.000286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a950c-94a5-4add-9567-b45477aaa67d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.012239 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1415/2848] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:32.016879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.027625 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d794c-2b87-456a-8e14-5ef5a719e0a7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.032933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.033873 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.034150 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.034425 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] PATH_INFO: `/v3/projects/5099eceece3848efb1dc79854816410a/users/c9fcabb3f02440f7aaf344b2bbaeffaa/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.052397 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5099eceece3848efb1dc79854816410a, user_id=c9fcabb3f02440f7aaf344b2bbaeffaa, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.055114 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.069080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1434/2849] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:32 2022] PUT /identity/v3/projects/5099eceece3848efb1dc79854816410a/users/c9fcabb3f02440f7aaf344b2bbaeffaa/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:32.081169 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.081453 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.081720 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.109884 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] MFA Rules not processed for user `c9fcabb3f02440f7aaf344b2bbaeffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:32.111232 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:32.115345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:32.181733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1416/2850] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:32.186978 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.224352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:32.266584 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68995632-d788-4a2b-8763-659dde1c9384 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.275132 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.276187 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.276989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.276989 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.277670 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.280279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.313454 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1435/2851] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:32 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:32.318551 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.330125 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9f3ac-a476-4b23-a091-1256d7563cfb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.412090 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.412924 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.413211 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.413486 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.414183 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.416297 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.438853 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1417/2852] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:32 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:32.443027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.454239 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e393-e868-4c94-962f-44057064f208 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.460746 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.461555 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.461933 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.462260 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.463356 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.465699 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.477388 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1436/2853] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:32.482288 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.492907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77735-8d8e-47e9-b50b-3fa1f77985a2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.498135 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.499153 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.499451 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.499733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] PATH_INFO: `/v3/projects/cc458125262a49ad850e326ed2c146f8/users/b6625cb60b7c44ab98df2c6d8112f0cb/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.515796 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc458125262a49ad850e326ed2c146f8, user_id=b6625cb60b7c44ab98df2c6d8112f0cb, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.518558 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.539781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1418/2854] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:32 2022] PUT /identity/v3/projects/cc458125262a49ad850e326ed2c146f8/users/b6625cb60b7c44ab98df2c6d8112f0cb/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:32.544117 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.554973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57940ea-650f-411f-956f-af30a795f22e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.623565 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.624253 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.624586 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.624864 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.625596 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.627262 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877c542-940a-4e5a-a2e6-e4834534d62f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.637654 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1437/2855] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:53:32.642549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.653893 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b925c-609d-4661-a7eb-fabe853122d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.659589 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.660503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.660784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.661146 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] PATH_INFO: `/v3/projects/cc458125262a49ad850e326ed2c146f8/users/b6625cb60b7c44ab98df2c6d8112f0cb/roles/5a49d18cafb842c69d51d5c93c8b2be9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.663782 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc458125262a49ad850e326ed2c146f8, user_id=b6625cb60b7c44ab98df2c6d8112f0cb, role_id=5a49d18cafb842c69d51d5c93c8b2be9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.666601 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.678753 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1419/2856] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:32 2022] PUT /identity/v3/projects/cc458125262a49ad850e326ed2c146f8/users/b6625cb60b7c44ab98df2c6d8112f0cb/roles/5a49d18cafb842c69d51d5c93c8b2be9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:32.689635 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.690019 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.690390 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.720000 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] MFA Rules not processed for user `b6625cb60b7c44ab98df2c6d8112f0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:32.721282 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:32.796464 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1438/2857] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:53:32.808339 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:32.815467 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:32.880534 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72898f-efc1-475a-bb54-7c577df78bec admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:32.885200 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:32.885953 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:32.886652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:32.886652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:32.898007 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:32.958410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:32.963415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:32.971032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1420/2858] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:36.659382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:36.660891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:36.675600 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc3ec-11e5-4823-a11e-f15cc845d913 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:36.681118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:36.682314 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:36.682687 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:36.682986 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:36.719428 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:36.756483 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:36.757988 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aec16-2f20-48f7-99ab-552a3b5341ad service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:36.767987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1439/2859] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:37.376285 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:37.385457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfc888-5e2e-406a-8ac3-5a517aacdd05 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:37.389823 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd8aa4f4a5cb441cae26c8dafa27548c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:37.390490 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:37.390769 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:37.391048 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:37.400861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:37.402313 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:37.412437 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1421/2860] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:53:39.164545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:39.165140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:39.165140 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:39.183924 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] MFA Rules not processed for user `565cc9705a94454b9e68595b85c6a286`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:39.185182 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:39.246034 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1440/2861] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:39.571353 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:39.573302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:39.640673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401eef-cd6b-4625-8195-ec5f5086dcc7 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:39.644784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:39.645442 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:39.645731 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:39.646008 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:39.647914 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:39.658082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:39.659577 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:39.668784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1422/2862] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:40.178240 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:40.179758 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:40.270642 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be364e9-fa85-4d1c-904f-0917c98c1419 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:40.275919 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:40.276591 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:40.276879 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:40.277160 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:40.278979 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:40.291232 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:40.292880 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f08731-fb07-4eca-a880-c46ef07233e0 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:40.303796 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1441/2863] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:41.350290 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:41.431046 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257264-318b-4ff5-923f-3498afe190d8 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:41.435567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:41.436926 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:41.437223 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:41.437494 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] PATH_INFO: `/v3/users/3c102f284b0b4ff99b333c3af860bd70` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:41.438798 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c102f284b0b4ff99b333c3af860bd70)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:41.440982 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:41.489095 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:41.492066 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c102f284b0b4ff99b333c3af860bd70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:41.501536 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c102f284b0b4ff99b333c3af860bd70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:41.510746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c102f284b0b4ff99b333c3af860bd70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:41.511859 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c102f284b0b4ff99b333c3af860bd70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:41.512841 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c102f284b0b4ff99b333c3af860bd70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:41.533231 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1423/2864] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:41 2022] DELETE /identity/v3/users/3c102f284b0b4ff99b333c3af860bd70 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:41.538192 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:41.538674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:41.538881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:41.558015 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:41.559265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:41.617846 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:41.629621 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1442/2865] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:41.640196 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:41.641697 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:41.726032 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041200d-a6db-4fbf-8d3d-b29783173155 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:41.730856 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:41.732001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:41.732363 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:41.732852 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:41.744727 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:41.746296 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:41.758590 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1424/2866] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:42.064643 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:42.066585 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:42.080660 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50b67a-b56f-40cc-963f-32f811ff7b98 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:42.087517 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:42.089956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:42.090306 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:42.090663 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] PATH_INFO: `/v3/projects/5491c649b1164ebf95ff8b8b7b289b87` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:42.092741 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5491c649b1164ebf95ff8b8b7b289b87)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:42.096148 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:42.162685 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5491c649b1164ebf95ff8b8b7b289b87', 'request_id': 'req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.173083 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5491c649b1164ebf95ff8b8b7b289b87', 'request_id': 'req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.182525 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5491c649b1164ebf95ff8b8b7b289b87', 'request_id': 'req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.195917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] The token cache is being invalidate because project 5491c649b1164ebf95ff8b8b7b289b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:42.196302 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.197070 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.198720 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1443/2867] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:42 2022] DELETE /identity/v3/projects/5491c649b1164ebf95ff8b8b7b289b87 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:42.202886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:42.213244 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0025820-61a1-4cae-9bcb-dcc5d8ff02dd service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:42.280045 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:42.281214 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:42.281609 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:42.282009 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] PATH_INFO: `/v3/users/c9fcabb3f02440f7aaf344b2bbaeffaa` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:42.285562 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fcabb3f02440f7aaf344b2bbaeffaa)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:42.287785 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:42.340533 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fcabb3f02440f7aaf344b2bbaeffaa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.348082 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fcabb3f02440f7aaf344b2bbaeffaa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.356650 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fcabb3f02440f7aaf344b2bbaeffaa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.357357 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fcabb3f02440f7aaf344b2bbaeffaa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.358439 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fcabb3f02440f7aaf344b2bbaeffaa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.361354 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:42.383139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1425/2868] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:42 2022] DELETE /identity/v3/users/c9fcabb3f02440f7aaf344b2bbaeffaa => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:42.725350 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:42.727220 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:42.746838 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a8e8-fd9a-4fce-a0c0-ed564ec843b8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:42.829444 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:42.830352 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:42.830633 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:42.830958 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] PATH_INFO: `/v3/projects/5099eceece3848efb1dc79854816410a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:42.832349 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5099eceece3848efb1dc79854816410a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:42.835674 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:42.930702 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5099eceece3848efb1dc79854816410a', 'request_id': 'req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.937713 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5099eceece3848efb1dc79854816410a', 'request_id': 'req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.947657 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5099eceece3848efb1dc79854816410a', 'request_id': 'req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.960611 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] The token cache is being invalidate because project 5099eceece3848efb1dc79854816410a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:42.961312 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.962389 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:42.964275 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1444/2869] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:42 2022] DELETE /identity/v3/projects/5099eceece3848efb1dc79854816410a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:42.968970 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:42.970424 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:42.984428 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4dbf3-d7d1-4acb-bb25-179a79fca416 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:43.051507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:43.052410 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.052693 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.052968 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] PATH_INFO: `/v3/users/b6625cb60b7c44ab98df2c6d8112f0cb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.054208 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6625cb60b7c44ab98df2c6d8112f0cb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:43.056426 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:43.127549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6625cb60b7c44ab98df2c6d8112f0cb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.135016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6625cb60b7c44ab98df2c6d8112f0cb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.145784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6625cb60b7c44ab98df2c6d8112f0cb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.146615 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6625cb60b7c44ab98df2c6d8112f0cb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.147362 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0e7b3be-e10b-47dc-aae0-647a4ae9de02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6625cb60b7c44ab98df2c6d8112f0cb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.167466 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1426/2870] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:53:42 2022] DELETE /identity/v3/users/b6625cb60b7c44ab98df2c6d8112f0cb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:43.471204 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:43.472745 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:43.487458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45dc73-4526-4105-84ca-74bfb4fbdc7d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:43.503184 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:43.564327 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:43.565238 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.565588 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.566592 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] PATH_INFO: `/v3/projects/cc458125262a49ad850e326ed2c146f8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.567875 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc458125262a49ad850e326ed2c146f8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:43.569844 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:43.626658 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc458125262a49ad850e326ed2c146f8', 'request_id': 'req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.633589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc458125262a49ad850e326ed2c146f8', 'request_id': 'req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.641891 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc458125262a49ad850e326ed2c146f8', 'request_id': 'req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.654803 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] The token cache is being invalidate because project cc458125262a49ad850e326ed2c146f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:53:43.655971 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.656639 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9981f6e7-b57b-43e2-b1a3-ff5f2d3082e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:53:43.658243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1445/2871] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:53:43 2022] DELETE /identity/v3/projects/cc458125262a49ad850e326ed2c146f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:43.663316 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.663833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.664094 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.684342 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:43.685550 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:43.758523 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1427/2872] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:43.766858 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.767418 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.767505 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.786474 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:43.787833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:43.795944 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1446/2873] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:43.800409 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:43.805392 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:43.813789 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795e3d3-b8ee-4f84-9e67-49f0a6bd1732 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:43.826293 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:43.827111 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.828001 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.828286 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.829234 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:53:43.829547 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:43.831287 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:43.846432 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1428/2874] 10.222.0.65 () {58 vars in 1152 bytes} [Fri Jun 24 09:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:53:43.851063 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:43.861534 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807df14-7830-49e4-b7de-45e21d5c50fb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:43.867889 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:43.868118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:43.868221 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:43.868524 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:43.869228 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:43.871172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:43.910283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1447/2875] 10.222.0.65 () {60 vars in 1152 bytes} [Fri Jun 24 09:53:43 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:43.914917 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:43.927321 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8e13-b7bb-4b26-bae0-d8d06dafb0ad admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:44.002136 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:44.002464 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.002745 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.003016 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.003748 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:44.005699 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:44.028591 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1429/2876] 10.222.0.65 () {60 vars in 1143 bytes} [Fri Jun 24 09:53:43 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:53:44.033029 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:44.044116 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302eacb-3754-40e6-b5e9-9b5b05407450 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:44.049241 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:44.049962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.050248 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.050529 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.051298 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:44.052976 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:44.066899 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1448/2877] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:44.068457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:44.078724 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458547-2aac-41e7-8f18-e58a8cd29554 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:44.083802 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:44.084476 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.084894 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.085170 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.085905 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:44.087548 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f417-1d1e-4d4d-8bde-6a0ad04aea9e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:44.098113 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1430/2878] 10.222.0.65 () {58 vars in 1121 bytes} [Fri Jun 24 09:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:53:44.102833 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:44.113305 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c1b6-ebf8-4787-bbe1-e09fe42f44c3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:44.119040 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:44.119987 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.120278 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.120562 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] PATH_INFO: `/v3/projects/30c82c90f8544e0994d5d09c849dbe58/users/2beac86a198a418eb74d6195596cd7e9/roles/58709cf0655a474eb4a33606ebb525df` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.136317 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=30c82c90f8544e0994d5d09c849dbe58, user_id=2beac86a198a418eb74d6195596cd7e9, role_id=58709cf0655a474eb4a33606ebb525df)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:44.139051 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:44.152843 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1449/2879] 10.222.0.65 () {60 vars in 1482 bytes} [Fri Jun 24 09:53:44 2022] PUT /identity/v3/projects/30c82c90f8544e0994d5d09c849dbe58/users/2beac86a198a418eb74d6195596cd7e9/roles/58709cf0655a474eb4a33606ebb525df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:53:44.164680 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.164969 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.165251 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.327864 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] MFA Rules not processed for user `2beac86a198a418eb74d6195596cd7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:53:44.329193 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:44.401206 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1431/2880] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:53:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:53:44.414185 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:44.416179 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:44.463594 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:44.480480 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5114-f2ac-4f8a-84ea-4eaa79f092ae admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:44.484560 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01aa5644b3834b0d887bbfc854ebc8cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:44.485270 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:44.485549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:44.485795 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:44.495319 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:44.496776 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:44.506107 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1450/2881] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:46.194661 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:46.196332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:46.271540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895a953-3905-4159-82e9-6901e8dcb240 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:46.275580 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:46.276265 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:46.276557 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:46.276898 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:46.279899 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:46.292866 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:46.350299 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:46.351804 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:46.362018 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1432/2882] 10.222.0.65 () {60 vars in 1396 bytes} [Fri Jun 24 09:53:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:52.980545 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:52.981540 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:53.078905 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a81c-7312-43bb-8a2b-35587d8a0c68 service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:53.085736 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:53.086436 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:53.086730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:53.087001 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:53.088792 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:53.101610 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:53.103279 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:53.114079 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1451/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:53:56.357257 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=22 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:53:57.305585 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:53:57.310784 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:57.332072 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1961f-bc16-4843-892c-e5cbe32e5f68 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:53:57.339795 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15d802693dbc490c890d922d838f856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:53:57.340777 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:53:57.341430 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:53:57.341881 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:53:57.344840 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:53:57.359364 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:53:57.362385 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:53:57.376644 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1433/2884] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:02.024382 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:02.026056 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:02.095730 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990258a-64d2-44f5-ba6f-87d633690211 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:02.101441 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:02.102399 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:02.102680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:02.102955 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] PATH_INFO: `/v3/users/b2c58a25e4cc44a8b1fd9e1edfabdc02` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:02.104310 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c58a25e4cc44a8b1fd9e1edfabdc02)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:02.106351 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:02.172080 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c58a25e4cc44a8b1fd9e1edfabdc02'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.185629 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c58a25e4cc44a8b1fd9e1edfabdc02'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.186276 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c58a25e4cc44a8b1fd9e1edfabdc02'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.189967 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:02.196344 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c58a25e4cc44a8b1fd9e1edfabdc02'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.205247 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c58a25e4cc44a8b1fd9e1edfabdc02'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.212536 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1452/2885] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:02 2022] DELETE /identity/v3/users/b2c58a25e4cc44a8b1fd9e1edfabdc02 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:02.409906 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:02.422040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee230aa4-8cfb-4a9e-bdba-c09b44e7be53 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:02.483457 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:02.484329 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:02.484608 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:02.484880 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] PATH_INFO: `/v3/projects/47cfee44c4264b0a8cfb5a93b7f5378a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:02.486043 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47cfee44c4264b0a8cfb5a93b7f5378a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:02.487972 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:02.535754 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47cfee44c4264b0a8cfb5a93b7f5378a', 'request_id': 'req-d132a42d-a75d-4be3-ac08-37969a959f8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.543779 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47cfee44c4264b0a8cfb5a93b7f5378a', 'request_id': 'req-d132a42d-a75d-4be3-ac08-37969a959f8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.549168 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47cfee44c4264b0a8cfb5a93b7f5378a', 'request_id': 'req-d132a42d-a75d-4be3-ac08-37969a959f8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.560218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] The token cache is being invalidate because project 47cfee44c4264b0a8cfb5a93b7f5378a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:02.560507 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.561225 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.562886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1434/2886] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:02 2022] DELETE /identity/v3/projects/47cfee44c4264b0a8cfb5a93b7f5378a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:02.567866 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:02.569414 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:02.583956 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec3f78-0e5e-4ab7-96d2-00c1f1080cca admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:02.644985 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:02.645881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:02.646159 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:02.646430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] PATH_INFO: `/v3/users/3677d760764d48e9be49b5bfa1fa8ba6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:02.647652 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] RBAC: Authorizing `identity:delete_user(user_id=3677d760764d48e9be49b5bfa1fa8ba6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:02.649622 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:02.696821 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3677d760764d48e9be49b5bfa1fa8ba6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.709062 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3677d760764d48e9be49b5bfa1fa8ba6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.709726 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3677d760764d48e9be49b5bfa1fa8ba6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.717468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3677d760764d48e9be49b5bfa1fa8ba6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.724900 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3677d760764d48e9be49b5bfa1fa8ba6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:02.730783 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1453/2887] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:02 2022] DELETE /identity/v3/users/3677d760764d48e9be49b5bfa1fa8ba6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:02.909255 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:02.911332 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:02.929975 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132a42d-a75d-4be3-ac08-37969a959f8c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:02.990696 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:02.991624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:02.991907 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:02.992306 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] PATH_INFO: `/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:02.993512 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab460c4e3e046fea8888e9c1bf79cb3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:02.995474 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:03.016681 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:03.043425 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab460c4e3e046fea8888e9c1bf79cb3', 'request_id': 'req-143c806d-df06-4aa1-a5d4-644f69388d89'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:03.050673 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab460c4e3e046fea8888e9c1bf79cb3', 'request_id': 'req-143c806d-df06-4aa1-a5d4-644f69388d89'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:03.056566 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab460c4e3e046fea8888e9c1bf79cb3', 'request_id': 'req-143c806d-df06-4aa1-a5d4-644f69388d89'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:03.067766 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] The token cache is being invalidate because project 8ab460c4e3e046fea8888e9c1bf79cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:03.068049 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:03.068652 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-143c806d-df06-4aa1-a5d4-644f69388d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:03.070233 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1435/2888] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:02 2022] DELETE /identity/v3/projects/8ab460c4e3e046fea8888e9c1bf79cb3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:12.727322 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:13.064611 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:14.888218 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:14.889962 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:14.904245 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319e8fe-4701-4a70-bf3c-48ec9aed12af admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:14.966983 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:14.967929 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:14.968216 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:14.968496 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] PATH_INFO: `/v3/users/2beac86a198a418eb74d6195596cd7e9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:14.969799 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2beac86a198a418eb74d6195596cd7e9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:14.971859 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:15.020273 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2beac86a198a418eb74d6195596cd7e9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.031712 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2beac86a198a418eb74d6195596cd7e9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.032395 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2beac86a198a418eb74d6195596cd7e9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.040452 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2beac86a198a418eb74d6195596cd7e9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.047328 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2beac86a198a418eb74d6195596cd7e9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.053475 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1454/2889] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:54:14 2022] DELETE /identity/v3/users/2beac86a198a418eb74d6195596cd7e9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:54:15.058422 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:15.058940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:15.058940 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:15.077447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] MFA Rules not processed for user `0fdc8f07afea4aa1a81df36db46bb225`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:54:15.078649 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:15.140345 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1436/2890] 10.222.0.65 () {58 vars in 957 bytes} [Fri Jun 24 09:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:54:15.150595 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:15.152180 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:15.214870 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b707c0f-c72c-499b-a3a6-08eb563e3dc7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:15.219384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:15.220053 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:15.220330 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:15.220604 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:15.231554 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:15.233025 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:15.243734 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1455/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:54:15.527833 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:54:15.529315 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:54:15.541931 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d6204-a3c6-414c-854a-baf4c48738c3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:54:15.547188 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:54:15.548100 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:54:15.548390 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:54:15.548672 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] PATH_INFO: `/v3/projects/30c82c90f8544e0994d5d09c849dbe58` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:54:15.549886 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=30c82c90f8544e0994d5d09c849dbe58)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:54:15.552033 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:54:15.602355 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30c82c90f8544e0994d5d09c849dbe58', 'request_id': 'req-c9905255-c2c1-4135-a635-3ca4dbeece5b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.611709 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30c82c90f8544e0994d5d09c849dbe58', 'request_id': 'req-c9905255-c2c1-4135-a635-3ca4dbeece5b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.618733 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30c82c90f8544e0994d5d09c849dbe58', 'request_id': 'req-c9905255-c2c1-4135-a635-3ca4dbeece5b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.632010 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] The token cache is being invalidate because project 30c82c90f8544e0994d5d09c849dbe58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:54:15.632310 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.632989 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:54:15.634549 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1437/2892] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:54:15 2022] DELETE /identity/v3/projects/30c82c90f8544e0994d5d09c849dbe58 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:54:25.238789 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:54:25.627978 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:37.831533 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:37.871098 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa161c52-c30e-46d2-9c8c-bcea2e2086de service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:37.921458 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2381343893e74c0a919ff4d0d59b655c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:37.922118 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:37.922400 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:37.922680 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:37.924549 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:38.057233 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:38.058743 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:38.078512 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1456/2893] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:41.315657 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:41.317263 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:41.448567 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905255-c2c1-4135-a635-3ca4dbeece5b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:41.454503 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:41.455122 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:41.455403 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:41.455678 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:41.458056 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:41.470160 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:41.471882 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:41.483218 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1438/2894] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:44.763266 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:44.765027 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:44.780430 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6007a06-6ef0-4cef-b86d-3d9ad8055dd8 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:44.786199 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:44.787137 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:44.787419 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:44.787691 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] PATH_INFO: `/v3/users/565cc9705a94454b9e68595b85c6a286` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:44.788993 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565cc9705a94454b9e68595b85c6a286)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:44.790937 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:44.818391 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:44.850471 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565cc9705a94454b9e68595b85c6a286'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:44.863575 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565cc9705a94454b9e68595b85c6a286'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:44.864295 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565cc9705a94454b9e68595b85c6a286'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:44.874917 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565cc9705a94454b9e68595b85c6a286'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:44.883509 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565cc9705a94454b9e68595b85c6a286'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:44.891848 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1457/2895] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:44 2022] DELETE /identity/v3/users/565cc9705a94454b9e68595b85c6a286 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:45.090790 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:45.102861 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d269392-98f3-4157-8874-3b4b890d46cb service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:45.166952 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:45.167858 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:45.168139 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:45.168415 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] PATH_INFO: `/v3/projects/86f042cb88b7426ea9733f70033b55e3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:45.169633 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f042cb88b7426ea9733f70033b55e3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:45.171624 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:45.219038 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:45.220497 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f042cb88b7426ea9733f70033b55e3', 'request_id': 'req-85b94992-a876-40ad-a371-c0f6fa818a60'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.229302 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f042cb88b7426ea9733f70033b55e3', 'request_id': 'req-85b94992-a876-40ad-a371-c0f6fa818a60'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.235780 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f042cb88b7426ea9733f70033b55e3', 'request_id': 'req-85b94992-a876-40ad-a371-c0f6fa818a60'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.247746 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] The token cache is being invalidate because project 86f042cb88b7426ea9733f70033b55e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:45.248040 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.248730 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.250341 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1439/2896] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:45 2022] DELETE /identity/v3/projects/86f042cb88b7426ea9733f70033b55e3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:45.254757 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:45.256384 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:45.272472 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d709da7-b10b-45ed-aa48-f958e256d441 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:45.342834 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:45.343973 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:45.344324 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:45.344684 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] PATH_INFO: `/v3/users/7568764c93b54e2496c947ed049f56af` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:45.346032 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7568764c93b54e2496c947ed049f56af)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:45.348468 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:45.404243 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7568764c93b54e2496c947ed049f56af'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.418589 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7568764c93b54e2496c947ed049f56af'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.419283 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7568764c93b54e2496c947ed049f56af'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.429147 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7568764c93b54e2496c947ed049f56af'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.438172 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7568764c93b54e2496c947ed049f56af'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.443851 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1458/2897] 10.222.0.65 () {58 vars in 1223 bytes} [Fri Jun 24 09:55:45 2022] DELETE /identity/v3/users/7568764c93b54e2496c947ed049f56af => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:45.631499 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:45.633054 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:45.650647 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b94992-a876-40ad-a371-c0f6fa818a60 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:45.714469 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdc8f07afea4aa1a81df36db46bb225', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46a0c5caeb0244aaa5e89844285d900b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:45.715518 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:45.715781 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:45.716061 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] PATH_INFO: `/v3/projects/896af774f0b14c7d8727c344e41537f8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:45.717266 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896af774f0b14c7d8727c344e41537f8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:45.719199 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:45.781996 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896af774f0b14c7d8727c344e41537f8', 'request_id': 'req-4035e408-e1eb-4fff-8f94-a34319730637'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.795106 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896af774f0b14c7d8727c344e41537f8', 'request_id': 'req-4035e408-e1eb-4fff-8f94-a34319730637'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.802637 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896af774f0b14c7d8727c344e41537f8', 'request_id': 'req-4035e408-e1eb-4fff-8f94-a34319730637'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.815447 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] The token cache is being invalidate because project 896af774f0b14c7d8727c344e41537f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:55:45.816262 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.816967 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4035e408-e1eb-4fff-8f94-a34319730637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:55:45.818540 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1440/2898] 10.222.0.65 () {58 vars in 1232 bytes} [Fri Jun 24 09:55:45 2022] DELETE /identity/v3/projects/896af774f0b14c7d8727c344e41537f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:55:54.818806 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:55:54.820553 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:54.843265 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:55:54.896274 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967e10-2720-44e0-abc5-f139bbb6b13d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:55:54.956376 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e945abf5978842a79cfaf92c479542a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f353bca7604872a4e99359eaeef3ff', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:55:54.957277 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:55:54.957582 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:55:54.957860 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:55:54.959881 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:55:54.971446 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:55:54.973346 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca9e9a-4742-44e4-b735-d997f389cf0a service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:55:54.983410 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1459/2899] 10.222.0.65 () {60 vars in 1365 bytes} [Fri Jun 24 09:55:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:55:55.811958 dv-cinder-803823-13-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:56:04.979139 dv-cinder-803823-13-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=50 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}